Merge lp:~jtv/maas/1.2-bug-1069734 into lp:maas/1.2

Proposed by Jeroen T. Vermeulen on 2012-10-25
Status: Merged
Approved by: Jeroen T. Vermeulen on 2012-10-26
Approved revision: 1273
Merged at revision: 1272
Proposed branch: lp:~jtv/maas/1.2-bug-1069734
Merge into: lp:maas/1.2
Diff against target: 1207 lines (+691/-306)
12 files modified
etc/cron.d/maas-gc (+2/-6)
setup.py (+0/-2)
src/maasserver/api.py (+2/-2)
src/maasserver/management/commands/gc.py (+0/-24)
src/maasserver/migrations/0039_add_filestorage_content.py (+233/-0)
src/maasserver/migrations/0040_make_filestorage_data_not_null.py (+203/-0)
src/maasserver/migrations/0041_remove_filestorage_data.py (+213/-0)
src/maasserver/models/filestorage.py (+15/-92)
src/maasserver/testing/factory.py (+4/-4)
src/maasserver/tests/test_api.py (+4/-2)
src/maasserver/tests/test_commands.py (+0/-11)
src/maasserver/tests/test_filestorage.py (+15/-163)
To merge this branch: bzr merge lp:~jtv/maas/1.2-bug-1069734
Reviewer Review Type Date Requested Status
Raphaël Badin (community) 2012-10-25 Approve on 2012-10-26
Review via email: mp+131334@code.launchpad.net

Commit message

Move file storage from the filesystem into the database, so that it's no longer local to a single app server.

Functionally the change is hidden inside FileStorage, but the need for garbage collection goes out the window as well. We should remove the maas-gc cron script altogether, but for that, the packaging branch needs updating as well. Keeping the file in disabled form for now decouples the two changes.

Description of the change

This is the change for the 1.2 branch. We'll want to apply the change to trunk as well.

Jeroen

To post a comment you must log in.
Raphaël Badin (rvb) wrote :

Looks generally good… but I'm wondering about how you tested this, hence "Needs information".

[0]

560 + rmtree(
561 + os.path.join(settings.MEDIA_ROOT, upload_dir),
562 + ignore_errors=True)

I think this is a bit dangerous, because it uses a user setting. Imagine the case where a user is installing MAAS (with a version which has this migration) but changes MEDIA_ROOT to a custom location. This migration, without any good reason, will issue a rm -rf ${MEDIA_ROOT}/storage. I really wonder if we shouldn't leave that directory alone. The flip side would be that old installations will have a 'storage' directory with a few files in it but new installations (which is probably what we should focus on) will be fine (and the rf -rf ... won't be run).

[1]

Testing: migrations cannot be unit tested but I'd be more comfortable with this change knowing that the migration has been tested on a instance with real data in FileStorage. How did you test this?

review: Needs Information
Raphaël Badin (rvb) wrote :

All right, thanks for the explanations about the manual testing you did. Maybe something we could do is create utilities methods in migrations (like you've done here) and test these… but I guess the utilities would have to be written in a very special way so that one could test them without calling the orm at all.

review: Approve
lp:~jtv/maas/1.2-bug-1069734 updated on 2012-10-26
1273. By Jeroen T. Vermeulen on 2012-10-26

Only remove old storage directory if MEDIA_ROOT is set, and exists.

Jeroen T. Vermeulen (jtv) wrote :

Thanks. Also as discussed, I've updated the rmtree code. The deletion no longer happens if MEDIA_ROOT is not set, or if it does not identify an existing directory.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'etc/cron.d/maas-gc'
2--- etc/cron.d/maas-gc 2012-08-03 16:33:05 +0000
3+++ etc/cron.d/maas-gc 2012-10-26 07:56:20 +0000
4@@ -1,8 +1,4 @@
5 # Perform daily background cleanups in MAAS.
6 #
7-# The "maas gc" command is for garbage-collection, such as deleting uploaded
8-# files from Juju's file storage API, and in the future commissioning logs,
9-# that have been superseded by newer ones. (This isn't done immediately
10-# when the files are overwritten because (1) the transaction that overwrites
11-# them may fail, and (2) a file may still be in use when it's overwritten.)
12-0 0 * * * root /usr/sbin/maas gc &> /dev/null
13+# This currently does nothing: maas-gc is no longer needed.
14+#0 0 * * * root /usr/sbin/maas gc &> /dev/null
15
16=== modified file 'setup.py'
17--- setup.py 2012-10-01 22:56:46 +0000
18+++ setup.py 2012-10-26 07:56:20 +0000
19@@ -65,8 +65,6 @@
20 'etc/maas/commissioning-user-data',
21 'contrib/maas-http.conf',
22 'contrib/maas_local_settings.py']),
23- ('/etc/cron.d',
24- ['etc/cron.d/maas-gc']),
25 ('/usr/share/maas',
26 ['contrib/wsgi.py',
27 'etc/celeryconfig.py',
28
29=== modified file 'src/maasserver/api.py'
30--- src/maasserver/api.py 2012-10-11 13:55:44 +0000
31+++ src/maasserver/api.py 2012-10-26 07:56:20 +0000
32@@ -87,7 +87,6 @@
33 from functools import partial
34 import httplib
35 from inspect import getdoc
36-import simplejson as json
37 import sys
38 from textwrap import dedent
39
40@@ -175,6 +174,7 @@
41 from piston.utils import rc
42 from provisioningserver.enum import POWER_TYPE
43 from provisioningserver.kernel_opts import KernelParameters
44+import simplejson as json
45
46
47 class OperationsResource(Resource):
48@@ -926,7 +926,7 @@
49 db_file = FileStorage.objects.get(filename=filename)
50 except FileStorage.DoesNotExist:
51 raise MAASAPINotFound("File not found")
52- return HttpResponse(db_file.data.read(), status=httplib.OK)
53+ return HttpResponse(db_file.content, status=httplib.OK)
54
55
56 class AnonFilesHandler(AnonymousOperationsHandler):
57
58=== removed file 'src/maasserver/management/commands/gc.py'
59--- src/maasserver/management/commands/gc.py 2012-04-16 10:00:51 +0000
60+++ src/maasserver/management/commands/gc.py 1970-01-01 00:00:00 +0000
61@@ -1,24 +0,0 @@
62-# Copyright 2012 Canonical Ltd. This software is licensed under the
63-# GNU Affero General Public License version 3 (see the file LICENSE).
64-
65-"""Custom django command: garabge-collect."""
66-
67-from __future__ import (
68- absolute_import,
69- print_function,
70- unicode_literals,
71- )
72-
73-__metaclass__ = type
74-__all__ = [
75- 'Command',
76- ]
77-
78-
79-from django.core.management.base import BaseCommand
80-from maasserver.models import FileStorage
81-
82-
83-class Command(BaseCommand):
84- def handle(self, *args, **options):
85- FileStorage.objects.collect_garbage()
86
87=== added file 'src/maasserver/migrations/0039_add_filestorage_content.py'
88--- src/maasserver/migrations/0039_add_filestorage_content.py 1970-01-01 00:00:00 +0000
89+++ src/maasserver/migrations/0039_add_filestorage_content.py 2012-10-26 07:56:20 +0000
90@@ -0,0 +1,233 @@
91+# -*- coding: utf-8 -*-
92+from base64 import b64encode
93+import datetime
94+import os.path
95+
96+from django.conf import settings
97+from django.db import models
98+from south.db import db
99+from south.v2 import SchemaMigration
100+
101+
102+def get_unmigrated_filestorages(orm):
103+ """Find FileStorage objects whose data needs migrating."""
104+ return orm['maasserver.FileStorage'].objects.filter(content=None)
105+
106+
107+def read_file(storage):
108+ """Read file contents from a FileStorage."""
109+ return storage.data.read()
110+
111+
112+def copy_files_into_database(orm):
113+ """Copy file contents into the "content" field."""
114+ for storage in get_unmigrated_filestorages(orm):
115+ raw_content = read_file(storage)
116+ storage.content = b64encode(raw_content).decode('ascii')
117+ storage.save()
118+
119+
120+class Migration(SchemaMigration):
121+
122+ def forwards(self, orm):
123+ # Adding field 'FileStorage.content'
124+ db.add_column(u'maasserver_filestorage', 'content',
125+ self.gf('metadataserver.fields.BinaryField')(null=True),
126+ keep_default=False)
127+
128+ # Changing field 'FileStorage.filename'
129+ db.alter_column(u'maasserver_filestorage', 'filename', self.gf('django.db.models.fields.CharField')(unique=True, max_length=255))
130+
131+ # Effecting data migration. Not deleting the old files yet; the
132+ # database transaction might still abort for whatever reason.
133+ copy_files_into_database(orm)
134+
135+ def backwards(self, orm):
136+ # Deleting field 'FileStorage.content'
137+ db.delete_column(u'maasserver_filestorage', 'content')
138+
139+
140+ # Changing field 'FileStorage.filename'
141+ db.alter_column(u'maasserver_filestorage', 'filename', self.gf('django.db.models.fields.CharField')(max_length=200, unique=True))
142+
143+ models = {
144+ 'auth.group': {
145+ 'Meta': {'object_name': 'Group'},
146+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
147+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
148+ 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
149+ },
150+ 'auth.permission': {
151+ 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
152+ 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
153+ 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
154+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
155+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
156+ },
157+ 'auth.user': {
158+ 'Meta': {'object_name': 'User'},
159+ 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
160+ 'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
161+ 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
162+ 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
163+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
164+ 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
165+ 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
166+ 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
167+ 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
168+ 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
169+ 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
170+ 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
171+ 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
172+ },
173+ 'contenttypes.contenttype': {
174+ 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
175+ 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
176+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
177+ 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
178+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
179+ },
180+ u'maasserver.bootimage': {
181+ 'Meta': {'unique_together': "((u'architecture', u'subarchitecture', u'release', u'purpose'),)", 'object_name': 'BootImage'},
182+ 'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
183+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
184+ 'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
185+ 'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
186+ 'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'})
187+ },
188+ u'maasserver.componenterror': {
189+ 'Meta': {'object_name': 'ComponentError'},
190+ 'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
191+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
192+ 'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
193+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
194+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
195+ },
196+ u'maasserver.config': {
197+ 'Meta': {'object_name': 'Config'},
198+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
199+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
200+ 'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
201+ },
202+ u'maasserver.dhcplease': {
203+ 'Meta': {'object_name': 'DHCPLease'},
204+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
205+ 'ip': ('django.db.models.fields.IPAddressField', [], {'unique': 'True', 'max_length': '15'}),
206+ 'mac': ('maasserver.fields.MACAddressField', [], {}),
207+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
208+ },
209+ u'maasserver.filestorage': {
210+ 'Meta': {'object_name': 'FileStorage'},
211+ 'content': ('metadataserver.fields.BinaryField', [], {'null': 'True'}),
212+ 'data': ('django.db.models.fields.files.FileField', [], {'max_length': '255'}),
213+ 'filename': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
214+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
215+ },
216+ u'maasserver.macaddress': {
217+ 'Meta': {'object_name': 'MACAddress'},
218+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
219+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
220+ 'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
221+ 'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
222+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
223+ },
224+ u'maasserver.node': {
225+ 'Meta': {'object_name': 'Node'},
226+ 'after_commissioning_action': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
227+ 'architecture': ('django.db.models.fields.CharField', [], {'default': "u'i386/generic'", 'max_length': '31'}),
228+ 'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
229+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
230+ 'distro_series': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '10', 'null': 'True', 'blank': 'True'}),
231+ 'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
232+ 'hardware_details': ('maasserver.fields.XMLField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
233+ 'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
234+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
235+ 'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
236+ 'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
237+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
238+ 'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'null': 'True', 'blank': 'True'}),
239+ 'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
240+ 'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
241+ 'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
242+ 'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-b588ce50-1ea0-11e2-946f-002608dc6120'", 'unique': 'True', 'max_length': '41'}),
243+ 'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
244+ 'token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'null': 'True'}),
245+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
246+ },
247+ u'maasserver.nodegroup': {
248+ 'Meta': {'object_name': 'NodeGroup'},
249+ 'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
250+ 'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'unique': 'True'}),
251+ 'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
252+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
253+ 'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
254+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
255+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
256+ 'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
257+ 'updated': ('django.db.models.fields.DateTimeField', [], {}),
258+ 'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
259+ },
260+ u'maasserver.nodegroupinterface': {
261+ 'Meta': {'unique_together': "((u'nodegroup', u'interface'),)", 'object_name': 'NodeGroupInterface'},
262+ 'broadcast_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
263+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
264+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
265+ 'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
266+ 'ip': ('django.db.models.fields.GenericIPAddressField', [], {'max_length': '39'}),
267+ 'ip_range_high': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
268+ 'ip_range_low': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
269+ 'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
270+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
271+ 'router_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
272+ 'subnet_mask': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
273+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
274+ },
275+ u'maasserver.sshkey': {
276+ 'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
277+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
278+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
279+ 'key': ('django.db.models.fields.TextField', [], {}),
280+ 'updated': ('django.db.models.fields.DateTimeField', [], {}),
281+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
282+ },
283+ u'maasserver.tag': {
284+ 'Meta': {'object_name': 'Tag'},
285+ 'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
286+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
287+ 'definition': ('django.db.models.fields.TextField', [], {}),
288+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
289+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
290+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
291+ },
292+ u'maasserver.userprofile': {
293+ 'Meta': {'object_name': 'UserProfile'},
294+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
295+ 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'})
296+ },
297+ 'piston.consumer': {
298+ 'Meta': {'object_name': 'Consumer'},
299+ 'description': ('django.db.models.fields.TextField', [], {}),
300+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
301+ 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
302+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
303+ 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
304+ 'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
305+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': "orm['auth.User']"})
306+ },
307+ 'piston.token': {
308+ 'Meta': {'object_name': 'Token'},
309+ 'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
310+ 'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
311+ 'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Consumer']"}),
312+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
313+ 'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
314+ 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
315+ 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
316+ 'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1351168636L'}),
317+ 'token_type': ('django.db.models.fields.IntegerField', [], {}),
318+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': "orm['auth.User']"}),
319+ 'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
320+ }
321+ }
322+
323+ complete_apps = ['maasserver']
324
325=== added file 'src/maasserver/migrations/0040_make_filestorage_data_not_null.py'
326--- src/maasserver/migrations/0040_make_filestorage_data_not_null.py 1970-01-01 00:00:00 +0000
327+++ src/maasserver/migrations/0040_make_filestorage_data_not_null.py 2012-10-26 07:56:20 +0000
328@@ -0,0 +1,203 @@
329+# -*- coding: utf-8 -*-
330+import datetime
331+
332+from django.db import models
333+from south.db import db
334+from south.v2 import SchemaMigration
335+
336+
337+class Migration(SchemaMigration):
338+
339+ def forwards(self, orm):
340+
341+ # Changing field 'FileStorage.content'
342+ # Disallow NULLs. The previous migration should have
343+ # initialized the column for all existing rows.
344+ db.alter_column(u'maasserver_filestorage', 'content', self.gf('metadataserver.fields.BinaryField')(null=False))
345+
346+ def backwards(self, orm):
347+
348+ # Changing field 'FileStorage.content'
349+ db.alter_column(u'maasserver_filestorage', 'content', self.gf('metadataserver.fields.BinaryField')(null=True))
350+
351+ models = {
352+ 'auth.group': {
353+ 'Meta': {'object_name': 'Group'},
354+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
355+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
356+ 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
357+ },
358+ 'auth.permission': {
359+ 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
360+ 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
361+ 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
362+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
363+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
364+ },
365+ 'auth.user': {
366+ 'Meta': {'object_name': 'User'},
367+ 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
368+ 'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
369+ 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
370+ 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
371+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
372+ 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
373+ 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
374+ 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
375+ 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
376+ 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
377+ 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
378+ 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
379+ 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
380+ },
381+ 'contenttypes.contenttype': {
382+ 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
383+ 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
384+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
385+ 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
386+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
387+ },
388+ u'maasserver.bootimage': {
389+ 'Meta': {'unique_together': "((u'architecture', u'subarchitecture', u'release', u'purpose'),)", 'object_name': 'BootImage'},
390+ 'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
391+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
392+ 'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
393+ 'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
394+ 'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'})
395+ },
396+ u'maasserver.componenterror': {
397+ 'Meta': {'object_name': 'ComponentError'},
398+ 'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
399+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
400+ 'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
401+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
402+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
403+ },
404+ u'maasserver.config': {
405+ 'Meta': {'object_name': 'Config'},
406+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
407+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
408+ 'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
409+ },
410+ u'maasserver.dhcplease': {
411+ 'Meta': {'object_name': 'DHCPLease'},
412+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
413+ 'ip': ('django.db.models.fields.IPAddressField', [], {'unique': 'True', 'max_length': '15'}),
414+ 'mac': ('maasserver.fields.MACAddressField', [], {}),
415+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
416+ },
417+ u'maasserver.filestorage': {
418+ 'Meta': {'object_name': 'FileStorage'},
419+ 'content': ('metadataserver.fields.BinaryField', [], {}),
420+ 'data': ('django.db.models.fields.files.FileField', [], {'max_length': '255'}),
421+ 'filename': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
422+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
423+ },
424+ u'maasserver.macaddress': {
425+ 'Meta': {'object_name': 'MACAddress'},
426+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
427+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
428+ 'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
429+ 'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
430+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
431+ },
432+ u'maasserver.node': {
433+ 'Meta': {'object_name': 'Node'},
434+ 'after_commissioning_action': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
435+ 'architecture': ('django.db.models.fields.CharField', [], {'default': "u'i386/generic'", 'max_length': '31'}),
436+ 'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
437+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
438+ 'distro_series': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '10', 'null': 'True', 'blank': 'True'}),
439+ 'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
440+ 'hardware_details': ('maasserver.fields.XMLField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
441+ 'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
442+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
443+ 'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
444+ 'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
445+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
446+ 'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'null': 'True', 'blank': 'True'}),
447+ 'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
448+ 'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
449+ 'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
450+ 'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-766d0f92-1ea5-11e2-9dee-002608dc6120'", 'unique': 'True', 'max_length': '41'}),
451+ 'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
452+ 'token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'null': 'True'}),
453+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
454+ },
455+ u'maasserver.nodegroup': {
456+ 'Meta': {'object_name': 'NodeGroup'},
457+ 'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
458+ 'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'unique': 'True'}),
459+ 'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
460+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
461+ 'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
462+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
463+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
464+ 'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
465+ 'updated': ('django.db.models.fields.DateTimeField', [], {}),
466+ 'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
467+ },
468+ u'maasserver.nodegroupinterface': {
469+ 'Meta': {'unique_together': "((u'nodegroup', u'interface'),)", 'object_name': 'NodeGroupInterface'},
470+ 'broadcast_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
471+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
472+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
473+ 'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
474+ 'ip': ('django.db.models.fields.GenericIPAddressField', [], {'max_length': '39'}),
475+ 'ip_range_high': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
476+ 'ip_range_low': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
477+ 'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
478+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
479+ 'router_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
480+ 'subnet_mask': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
481+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
482+ },
483+ u'maasserver.sshkey': {
484+ 'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
485+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
486+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
487+ 'key': ('django.db.models.fields.TextField', [], {}),
488+ 'updated': ('django.db.models.fields.DateTimeField', [], {}),
489+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
490+ },
491+ u'maasserver.tag': {
492+ 'Meta': {'object_name': 'Tag'},
493+ 'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
494+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
495+ 'definition': ('django.db.models.fields.TextField', [], {}),
496+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
497+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
498+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
499+ },
500+ u'maasserver.userprofile': {
501+ 'Meta': {'object_name': 'UserProfile'},
502+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
503+ 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'})
504+ },
505+ 'piston.consumer': {
506+ 'Meta': {'object_name': 'Consumer'},
507+ 'description': ('django.db.models.fields.TextField', [], {}),
508+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
509+ 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
510+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
511+ 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
512+ 'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
513+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': "orm['auth.User']"})
514+ },
515+ 'piston.token': {
516+ 'Meta': {'object_name': 'Token'},
517+ 'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
518+ 'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
519+ 'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Consumer']"}),
520+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
521+ 'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
522+ 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
523+ 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
524+ 'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1351170650L'}),
525+ 'token_type': ('django.db.models.fields.IntegerField', [], {}),
526+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': "orm['auth.User']"}),
527+ 'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
528+ }
529+ }
530+
531+ complete_apps = ['maasserver']
532
533=== added file 'src/maasserver/migrations/0041_remove_filestorage_data.py'
534--- src/maasserver/migrations/0041_remove_filestorage_data.py 1970-01-01 00:00:00 +0000
535+++ src/maasserver/migrations/0041_remove_filestorage_data.py 2012-10-26 07:56:20 +0000
536@@ -0,0 +1,213 @@
537+# -*- coding: utf-8 -*-
538+import datetime
539+import os.path
540+from shutil import rmtree
541+
542+from django.conf import settings
543+from django.db import models
544+from south.db import db
545+from south.v2 import SchemaMigration
546+
547+# The sub-directory of MEDIA_ROOT where FileStorage used to store its
548+# files. This duplicates the value of the now-removed
549+# FileStorage.upload_dir.
550+upload_dir = 'storage'
551+
552+
553+class Migration(SchemaMigration):
554+
555+ def forwards(self, orm):
556+ # Deleting field 'FileStorage.data'
557+ db.delete_column(u'maasserver_filestorage', 'data')
558+
559+ # Cleaning up any obsolete FileStorage files.
560+ if settings.MEDIA_ROOT and os.path.isdir(settings.MEDIA_ROOT):
561+ rmtree(
562+ os.path.join(settings.MEDIA_ROOT, upload_dir),
563+ ignore_errors=True)
564+
565+ def backwards(self, orm):
566+
567+ # User chose to not deal with backwards NULL issues for 'FileStorage.data'
568+ raise RuntimeError("Cannot reverse this migration. 'FileStorage.data' and its values cannot be restored.")
569+
570+ models = {
571+ 'auth.group': {
572+ 'Meta': {'object_name': 'Group'},
573+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
574+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
575+ 'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
576+ },
577+ 'auth.permission': {
578+ 'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
579+ 'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
580+ 'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
581+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
582+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
583+ },
584+ 'auth.user': {
585+ 'Meta': {'object_name': 'User'},
586+ 'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
587+ 'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
588+ 'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
589+ 'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
590+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
591+ 'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
592+ 'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
593+ 'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
594+ 'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
595+ 'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
596+ 'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
597+ 'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
598+ 'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
599+ },
600+ 'contenttypes.contenttype': {
601+ 'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
602+ 'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
603+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
604+ 'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
605+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
606+ },
607+ u'maasserver.bootimage': {
608+ 'Meta': {'unique_together': "((u'architecture', u'subarchitecture', u'release', u'purpose'),)", 'object_name': 'BootImage'},
609+ 'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
610+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
611+ 'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
612+ 'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
613+ 'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'})
614+ },
615+ u'maasserver.componenterror': {
616+ 'Meta': {'object_name': 'ComponentError'},
617+ 'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
618+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
619+ 'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
620+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
621+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
622+ },
623+ u'maasserver.config': {
624+ 'Meta': {'object_name': 'Config'},
625+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
626+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
627+ 'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
628+ },
629+ u'maasserver.dhcplease': {
630+ 'Meta': {'object_name': 'DHCPLease'},
631+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
632+ 'ip': ('django.db.models.fields.IPAddressField', [], {'unique': 'True', 'max_length': '15'}),
633+ 'mac': ('maasserver.fields.MACAddressField', [], {}),
634+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
635+ },
636+ u'maasserver.filestorage': {
637+ 'Meta': {'object_name': 'FileStorage'},
638+ 'content': ('metadataserver.fields.BinaryField', [], {}),
639+ 'filename': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
640+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
641+ },
642+ u'maasserver.macaddress': {
643+ 'Meta': {'object_name': 'MACAddress'},
644+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
645+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
646+ 'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
647+ 'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
648+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
649+ },
650+ u'maasserver.node': {
651+ 'Meta': {'object_name': 'Node'},
652+ 'after_commissioning_action': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
653+ 'architecture': ('django.db.models.fields.CharField', [], {'default': "u'i386/generic'", 'max_length': '31'}),
654+ 'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
655+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
656+ 'distro_series': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '10', 'null': 'True', 'blank': 'True'}),
657+ 'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
658+ 'hardware_details': ('maasserver.fields.XMLField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
659+ 'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
660+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
661+ 'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
662+ 'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
663+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
664+ 'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['auth.User']", 'null': 'True', 'blank': 'True'}),
665+ 'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
666+ 'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
667+ 'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
668+ 'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-da71dfcc-1ea5-11e2-8763-002608dc6120'", 'unique': 'True', 'max_length': '41'}),
669+ 'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
670+ 'token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'null': 'True'}),
671+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
672+ },
673+ u'maasserver.nodegroup': {
674+ 'Meta': {'object_name': 'NodeGroup'},
675+ 'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
676+ 'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Token']", 'unique': 'True'}),
677+ 'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
678+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
679+ 'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
680+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
681+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
682+ 'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
683+ 'updated': ('django.db.models.fields.DateTimeField', [], {}),
684+ 'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
685+ },
686+ u'maasserver.nodegroupinterface': {
687+ 'Meta': {'unique_together': "((u'nodegroup', u'interface'),)", 'object_name': 'NodeGroupInterface'},
688+ 'broadcast_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
689+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
690+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
691+ 'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
692+ 'ip': ('django.db.models.fields.GenericIPAddressField', [], {'max_length': '39'}),
693+ 'ip_range_high': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
694+ 'ip_range_low': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
695+ 'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
696+ 'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
697+ 'router_ip': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
698+ 'subnet_mask': ('django.db.models.fields.GenericIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
699+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
700+ },
701+ u'maasserver.sshkey': {
702+ 'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
703+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
704+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
705+ 'key': ('django.db.models.fields.TextField', [], {}),
706+ 'updated': ('django.db.models.fields.DateTimeField', [], {}),
707+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
708+ },
709+ u'maasserver.tag': {
710+ 'Meta': {'object_name': 'Tag'},
711+ 'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
712+ 'created': ('django.db.models.fields.DateTimeField', [], {}),
713+ 'definition': ('django.db.models.fields.TextField', [], {}),
714+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
715+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
716+ 'updated': ('django.db.models.fields.DateTimeField', [], {})
717+ },
718+ u'maasserver.userprofile': {
719+ 'Meta': {'object_name': 'UserProfile'},
720+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
721+ 'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'})
722+ },
723+ 'piston.consumer': {
724+ 'Meta': {'object_name': 'Consumer'},
725+ 'description': ('django.db.models.fields.TextField', [], {}),
726+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
727+ 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
728+ 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
729+ 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
730+ 'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
731+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': "orm['auth.User']"})
732+ },
733+ 'piston.token': {
734+ 'Meta': {'object_name': 'Token'},
735+ 'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
736+ 'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
737+ 'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['piston.Consumer']"}),
738+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
739+ 'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
740+ 'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
741+ 'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
742+ 'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1351170840L'}),
743+ 'token_type': ('django.db.models.fields.IntegerField', [], {}),
744+ 'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': "orm['auth.User']"}),
745+ 'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
746+ }
747+ }
748+
749+ complete_apps = ['maasserver']
750
751=== modified file 'src/maasserver/models/filestorage.py'
752--- src/maasserver/models/filestorage.py 2012-08-24 10:28:29 +0000
753+++ src/maasserver/models/filestorage.py 2012-10-26 07:56:20 +0000
754@@ -15,22 +15,17 @@
755 ]
756
757
758-from errno import ENOENT
759-import os
760-import time
761-
762-from django.conf import settings
763-from django.core.files.base import ContentFile
764-from django.core.files.storage import FileSystemStorage
765 from django.db.models import (
766 CharField,
767- FileField,
768 Manager,
769 Model,
770 )
771 from maasserver import DefaultMeta
772 from maasserver.models.cleansave import CleanSave
773-from maasserver.utils.orm import get_one
774+from metadataserver.fields import (
775+ Bin,
776+ BinaryField,
777+ )
778
779
780 class FileStorageManager(Manager):
781@@ -47,108 +42,36 @@
782 original file will not be affected. Also, any ongoing reads from the
783 old file will continue without iterruption.
784 """
785- # The time, in seconds, that an unreferenced file is allowed to
786- # persist in order to satisfy ongoing requests.
787- grace_time = 12 * 60 * 60
788-
789- def get_existing_storage(self, filename):
790- """Return an existing `FileStorage` of this name, or None."""
791- return get_one(self.filter(filename=filename))
792
793 def save_file(self, filename, file_object):
794- """Save the file to the filesystem and persist to the database.
795-
796- The file will end up in MEDIA_ROOT/storage/
797+ """Save the file to the database.
798
799 If a file of that name already existed, it will be replaced by the
800 new contents.
801 """
802 # This probably ought to read in chunks but large files are
803- # not expected. Also note that uploading a file with the same
804- # name as an existing one will cause that file to be written
805- # with a new generated name, and the old one remains where it
806- # is. See https://code.djangoproject.com/ticket/6157 - the
807- # Django devs consider deleting things dangerous ... ha.
808- # HOWEVER - this operation would need to be atomic anyway so
809- # it's safest left how it is for now (reads can overlap with
810- # writes from Juju).
811- content = ContentFile(file_object.read())
812-
813- storage = self.get_existing_storage(filename)
814- if storage is None:
815- storage = FileStorage(filename=filename)
816- storage.data.save(filename, content)
817+ # not expected.
818+ content = Bin(file_object.read())
819+ storage, created = self.get_or_create(
820+ filename=filename, defaults={'content': content})
821+ if not created:
822+ storage.content = content
823+ storage.save()
824 return storage
825
826- def list_stored_files(self):
827- """Find the files stored in the filesystem."""
828- dirs, files = FileStorage.storage.listdir(FileStorage.upload_dir)
829- return [
830- os.path.join(FileStorage.upload_dir, filename)
831- for filename in files]
832-
833- def list_referenced_files(self):
834- """Find the names of files that are referenced from `FileStorage`.
835-
836- :return: All file paths within MEDIA ROOT (relative to MEDIA_ROOT)
837- that have `FileStorage` entries referencing them.
838- :rtype: frozenset
839- """
840- return frozenset(
841- file_storage.data.name
842- for file_storage in self.all())
843-
844- def is_old(self, storage_filename):
845- """Is the named file in the filesystem storage old enough to be dead?
846-
847- :param storage_filename: The name under which the file is stored in
848- the filesystem, relative to MEDIA_ROOT. This need not be the
849- same name as its filename as stored in the `FileStorage` object.
850- It includes the name of the upload directory.
851- """
852- file_path = os.path.join(settings.MEDIA_ROOT, storage_filename)
853- mtime = os.stat(file_path).st_mtime
854- expiry = mtime + self.grace_time
855- return expiry <= time.time()
856-
857- def collect_garbage(self):
858- """Clean up stored files that are no longer accessible."""
859- # Avoid circular imports.
860- from maasserver.models import logger
861-
862- try:
863- stored_files = self.list_stored_files()
864- except OSError as e:
865- if e.errno != ENOENT:
866- raise
867- logger.info(
868- "Upload directory does not exist yet. "
869- "Skipping garbage collection.")
870- return
871- referenced_files = self.list_referenced_files()
872- for path in stored_files:
873- if path not in referenced_files and self.is_old(path):
874- FileStorage.storage.delete(path)
875-
876
877 class FileStorage(CleanSave, Model):
878 """A simple file storage keyed on file name.
879
880 :ivar filename: A unique file name to use for the data being stored.
881- :ivar data: The file's actual data.
882+ :ivar content: The file's actual data.
883 """
884
885 class Meta(DefaultMeta):
886 """Needed for South to recognize this model."""
887
888- storage = FileSystemStorage()
889-
890- upload_dir = "storage"
891-
892- # Unix filenames can be longer than this (e.g. 255 bytes), but leave
893- # some extra room for the full path, as well as a versioning suffix.
894- filename = CharField(max_length=200, unique=True, editable=False)
895- data = FileField(upload_to=upload_dir, storage=storage, max_length=255)
896+ filename = CharField(max_length=255, unique=True, editable=False)
897+ content = BinaryField(null=False)
898
899 objects = FileStorageManager()
900
901
902=== modified file 'src/maasserver/testing/factory.py'
903--- src/maasserver/testing/factory.py 2012-10-02 22:28:07 +0000
904+++ src/maasserver/testing/factory.py 2012-10-26 07:56:20 +0000
905@@ -296,13 +296,13 @@
906 admin.save()
907 return admin
908
909- def make_file_storage(self, filename=None, data=None):
910+ def make_file_storage(self, filename=None, content=None):
911 if filename is None:
912 filename = self.getRandomString(100)
913- if data is None:
914- data = self.getRandomString(1024).encode('ascii')
915+ if content is None:
916+ content = self.getRandomString(1024).encode('ascii')
917
918- return FileStorage.objects.save_file(filename, BytesIO(data))
919+ return FileStorage.objects.save_file(filename, BytesIO(content))
920
921 def make_oauth_header(self, **kwargs):
922 """Fake an OAuth authorization header.
923
924=== modified file 'src/maasserver/tests/test_api.py'
925--- src/maasserver/tests/test_api.py 2012-10-11 13:41:57 +0000
926+++ src/maasserver/tests/test_api.py 2012-10-26 07:56:20 +0000
927@@ -2383,7 +2383,8 @@
928 class AnonymousFileStorageAPITest(FileStorageAPITestMixin, AnonAPITestCase):
929
930 def test_get_works_anonymously(self):
931- factory.make_file_storage(filename="foofilers", data=b"give me rope")
932+ factory.make_file_storage(
933+ filename="foofilers", content=b"give me rope")
934 response = self.make_API_GET_request("get", "foofilers")
935
936 self.assertEqual(httplib.OK, response.status_code)
937@@ -2453,7 +2454,8 @@
938 self.assertEqual("file two", response.content)
939
940 def test_get_file_succeeds(self):
941- factory.make_file_storage(filename="foofilers", data=b"give me rope")
942+ factory.make_file_storage(
943+ filename="foofilers", content=b"give me rope")
944 response = self.make_API_GET_request("get", "foofilers")
945
946 self.assertEqual(httplib.OK, response.status_code)
947
948=== modified file 'src/maasserver/tests/test_commands.py'
949--- src/maasserver/tests/test_commands.py 2012-08-24 10:28:29 +0000
950+++ src/maasserver/tests/test_commands.py 2012-10-26 07:56:20 +0000
951@@ -14,13 +14,10 @@
952
953 from codecs import getwriter
954 from io import BytesIO
955-import os
956
957-from django.conf import settings
958 from django.contrib.auth.models import User
959 from django.core.cache import cache
960 from django.core.management import call_command
961-from maasserver.models import FileStorage
962 from maasserver.testing.factory import factory
963 from maasserver.utils.orm import get_one
964 from maastesting.djangotestcase import DjangoTestCase
965@@ -33,14 +30,6 @@
966 in a command's code, it should be extracted and unit-tested separately.
967 """
968
969- def test_gc(self):
970- upload_dir = os.path.join(settings.MEDIA_ROOT, FileStorage.upload_dir)
971- os.makedirs(upload_dir)
972- self.addCleanup(os.removedirs, upload_dir)
973- call_command('gc')
974- # The test is that we get here without errors.
975- pass
976-
977 def test_generate_api_doc(self):
978 out = BytesIO()
979 stdout = getwriter("UTF-8")(out)
980
981=== modified file 'src/maasserver/tests/test_filestorage.py'
982--- src/maasserver/tests/test_filestorage.py 2012-06-25 09:03:14 +0000
983+++ src/maasserver/tests/test_filestorage.py 2012-10-26 07:56:20 +0000
984@@ -14,45 +14,15 @@
985
986 import codecs
987 from io import BytesIO
988-import os
989-import shutil
990
991-from django.conf import settings
992 from maasserver.models import FileStorage
993 from maasserver.testing.factory import factory
994 from maasserver.testing.testcase import TestCase
995-from maastesting.utils import age_file
996-from testtools.matchers import (
997- GreaterThan,
998- LessThan,
999- )
1000
1001
1002 class FileStorageTest(TestCase):
1003 """Testing of the :class:`FileStorage` model."""
1004
1005- def make_upload_dir(self):
1006- """Create the upload directory, and arrange for eventual deletion.
1007-
1008- The directory must not already exist. If it does, this method will
1009- fail rather than arrange for deletion of a directory that may
1010- contain meaningful data.
1011-
1012- :return: Absolute path to the `FileStorage` upload directory. This
1013- is the directory where the actual files are stored.
1014- """
1015- media_root = settings.MEDIA_ROOT
1016- self.assertFalse(os.path.exists(media_root), "See media/README")
1017- self.addCleanup(shutil.rmtree, media_root, ignore_errors=True)
1018- os.mkdir(media_root)
1019- upload_dir = os.path.join(media_root, FileStorage.upload_dir)
1020- os.mkdir(upload_dir)
1021- return upload_dir
1022-
1023- def get_media_path(self, filename):
1024- """Get the path to a given stored file, relative to MEDIA_ROOT."""
1025- return os.path.join(FileStorage.upload_dir, filename)
1026-
1027 def make_data(self, including_text='data'):
1028 """Return arbitrary data.
1029
1030@@ -67,40 +37,24 @@
1031 text = "%s %s" % (including_text, factory.getRandomString())
1032 return text.encode('ascii')
1033
1034- def test_get_existing_storage_returns_None_if_none_found(self):
1035- nonexistent_file = factory.getRandomString()
1036- self.assertIsNone(
1037- FileStorage.objects.get_existing_storage(nonexistent_file))
1038-
1039- def test_get_existing_storage_finds_FileStorage(self):
1040- self.make_upload_dir()
1041- storage = factory.make_file_storage()
1042- self.assertEqual(
1043- storage,
1044- FileStorage.objects.get_existing_storage(storage.filename))
1045-
1046 def test_save_file_creates_storage(self):
1047- self.make_upload_dir()
1048 filename = factory.getRandomString()
1049- data = self.make_data()
1050- storage = FileStorage.objects.save_file(filename, BytesIO(data))
1051+ content = self.make_data()
1052+ storage = FileStorage.objects.save_file(filename, BytesIO(content))
1053 self.assertEqual(
1054- (filename, data),
1055- (storage.filename, storage.data.read()))
1056+ (filename, content),
1057+ (storage.filename, storage.content))
1058
1059 def test_storage_can_be_retrieved(self):
1060- self.make_upload_dir()
1061 filename = factory.getRandomString()
1062- data = self.make_data()
1063- factory.make_file_storage(filename=filename, data=data)
1064+ content = self.make_data()
1065+ factory.make_file_storage(filename=filename, content=content)
1066 storage = FileStorage.objects.get(filename=filename)
1067 self.assertEqual(
1068- (filename, data),
1069- (storage.filename, storage.data.read()))
1070+ (filename, content),
1071+ (storage.filename, storage.content))
1072
1073 def test_stores_binary_data(self):
1074- self.make_upload_dir()
1075-
1076 # This horrible binary data could never, ever, under any
1077 # encoding known to man be interpreted as text(1). Switch the
1078 # bytes of the byte-order mark around and by design you get an
1079@@ -114,121 +68,19 @@
1080
1081 # And yet, because FileStorage supports binary data, it comes
1082 # out intact.
1083- storage = factory.make_file_storage(filename="x", data=binary_data)
1084- self.assertEqual(binary_data, storage.data.read())
1085+ storage = factory.make_file_storage(filename="x", content=binary_data)
1086+ self.assertEqual(binary_data, storage.content)
1087
1088 def test_overwrites_file(self):
1089 # If a file of the same name has already been stored, the
1090 # reference to the old data gets overwritten with one to the new
1091- # data. They are actually different files on the filesystem.
1092- self.make_upload_dir()
1093+ # data.
1094 filename = factory.make_name('filename')
1095 old_storage = factory.make_file_storage(
1096- filename=filename, data=self.make_data('old data'))
1097+ filename=filename, content=self.make_data('old data'))
1098 new_data = self.make_data('new-data')
1099 new_storage = factory.make_file_storage(
1100- filename=filename, data=new_data)
1101- self.assertNotEqual(old_storage.data.name, new_storage.data.name)
1102+ filename=filename, content=new_data)
1103+ self.assertEqual(old_storage.filename, new_storage.filename)
1104 self.assertEqual(
1105- new_data, FileStorage.objects.get(filename=filename).data.read())
1106-
1107- def test_list_stored_files_lists_files(self):
1108- filename = factory.getRandomString()
1109- factory.make_file(
1110- location=self.make_upload_dir(), name=filename,
1111- contents=self.make_data())
1112- self.assertIn(
1113- self.get_media_path(filename),
1114- FileStorage.objects.list_stored_files())
1115-
1116- def test_list_stored_files_includes_referenced_files(self):
1117- self.make_upload_dir()
1118- storage = factory.make_file_storage()
1119- self.assertIn(
1120- storage.data.name, FileStorage.objects.list_stored_files())
1121-
1122- def test_list_referenced_files_lists_FileStorage_files(self):
1123- self.make_upload_dir()
1124- storage = factory.make_file_storage()
1125- self.assertIn(
1126- storage.data.name, FileStorage.objects.list_referenced_files())
1127-
1128- def test_list_referenced_files_excludes_unreferenced_files(self):
1129- filename = factory.getRandomString()
1130- factory.make_file(
1131- location=self.make_upload_dir(), name=filename,
1132- contents=self.make_data())
1133- self.assertNotIn(
1134- self.get_media_path(filename),
1135- FileStorage.objects.list_referenced_files())
1136-
1137- def test_list_referenced_files_uses_file_name_not_FileStorage_name(self):
1138- self.make_upload_dir()
1139- filename = factory.getRandomString()
1140- # The filename we're going to use is already taken. The file
1141- # we'll be looking at will have to have a different name.
1142- factory.make_file_storage(filename=filename)
1143- storage = factory.make_file_storage(filename=filename)
1144- # It's the name of the file, not the FileStorage.filename, that
1145- # is in list_referenced_files.
1146- self.assertIn(
1147- storage.data.name, FileStorage.objects.list_referenced_files())
1148-
1149- def test_is_old_returns_False_for_recent_file(self):
1150- filename = factory.getRandomString()
1151- path = factory.make_file(
1152- location=self.make_upload_dir(), name=filename,
1153- contents=self.make_data())
1154- age_file(path, FileStorage.objects.grace_time - 60)
1155- self.assertFalse(
1156- FileStorage.objects.is_old(self.get_media_path(filename)))
1157-
1158- def test_is_old_returns_True_for_old_file(self):
1159- filename = factory.getRandomString()
1160- path = factory.make_file(
1161- location=self.make_upload_dir(), name=filename,
1162- contents=self.make_data())
1163- age_file(path, FileStorage.objects.grace_time + 1)
1164- self.assertTrue(
1165- FileStorage.objects.is_old(self.get_media_path(filename)))
1166-
1167- def test_collect_garbage_deletes_garbage(self):
1168- filename = factory.getRandomString()
1169- path = factory.make_file(
1170- location=self.make_upload_dir(), name=filename,
1171- contents=self.make_data())
1172- age_file(path, FileStorage.objects.grace_time + 1)
1173- FileStorage.objects.collect_garbage()
1174- self.assertFalse(
1175- FileStorage.storage.exists(self.get_media_path(filename)))
1176-
1177- def test_grace_time_is_generous_but_not_unlimited(self):
1178- # Grace time for garbage collection is long enough that it won't
1179- # expire while the request that wrote it is still being handled.
1180- # But it won't keep a file around for ages. For instance, it'll
1181- # be more than 20 seconds, but less than a day.
1182- self.assertThat(FileStorage.objects.grace_time, GreaterThan(20))
1183- self.assertThat(FileStorage.objects.grace_time, LessThan(24 * 60 * 60))
1184-
1185- def test_collect_garbage_leaves_recent_files_alone(self):
1186- filename = factory.getRandomString()
1187- factory.make_file(
1188- location=self.make_upload_dir(), name=filename,
1189- contents=self.make_data())
1190- FileStorage.objects.collect_garbage()
1191- self.assertTrue(
1192- FileStorage.storage.exists(self.get_media_path(filename)))
1193-
1194- def test_collect_garbage_leaves_referenced_files_alone(self):
1195- self.make_upload_dir()
1196- storage = factory.make_file_storage()
1197- age_file(storage.data.path, FileStorage.objects.grace_time + 1)
1198- FileStorage.objects.collect_garbage()
1199- self.assertTrue(FileStorage.storage.exists(storage.data.name))
1200-
1201- def test_collect_garbage_tolerates_missing_upload_dir(self):
1202- # When MAAS is freshly installed, the upload directory is still
1203- # missing. But...
1204- FileStorage.objects.collect_garbage()
1205- # ...we get through garbage collection without breakage.
1206- pass
1207+ new_data, FileStorage.objects.get(filename=filename).content)

Subscribers

People subscribed via source and target branches

to status/vote changes: