Merge lp:~blake-rouse/maas/largefile-model into lp:~maas-committers/maas/trunk

[Blake Rouse]

Added LargeFile model, that will be used by the incoming BootResource model.

This will be used by the BootResource models to hold the data for the boot images. This design allows multiple files from simplestreams to reference the same content, which is almost always the case for "boot-kernel" and "di-kernel".

Also provides the ability to get the current progress of the file being stored. This will allow the UI to show progress will downloading the boot images.

[Jason Hobbs (jason-hobbs)]

This mostly looks good - I just have a few questions inline, and some minor things to change. If you have good answers, I'll approve.

review: Needs Information

[Julian Edwards (julian-edwards)]

On Thursday 07 Aug 2014 14:37:22 you wrote:
> Also provides the ability to get the current progress of the file being
> stored. This will allow the UI to show progress will downloading the boot
> images.

Just a small point, but see src/maasserver/models/downloadprogress.py and
consider whether you need to use that or deprecate it (it's not complete
IIRC).

[Jason Hobbs (jason-hobbs)]

On Thu, Aug 7, 2014 at 9:51 PM, Blake Rouse <email address hidden>
wrote:

> '\0' terminating character.
>
> > + ('sha256', self.gf('django.db.models.fields.CharField')(unique=True,
> max_length=65)),
>

What '\0' terminating character? This isn't C...

Jason

[Graham Binns (gmb)]

Thanks for this branch Blake; couple of nitpicks from me, but no blockers.

review: Approve

[Jason Hobbs (jason-hobbs)]

This branch should not land with the sha256sum field size 65. It should
definitely be 64.

On Fri, Aug 8, 2014 at 12:46 PM, Graham Binns <email address hidden>
wrote:

> Review: Approve
>
> Thanks for this branch Blake; couple of nitpicks from me, but no blockers.
>
> Diff comments:
>
> > === added file 'src/maasserver/migrations/0097_add_largefile_model.py'
> > --- src/maasserver/migrations/0097_add_largefile_model.py 1970-01-01
> 00:00:00 +0000
> > +++ src/maasserver/migrations/0097_add_largefile_model.py 2014-08-07
> 19:58:22 +0000
> > @@ -0,0 +1,344 @@
> > +from django.db import models
> > +from south.db import db
> > +# -*- coding: utf-8 -*-
> > +from south.utils import datetime_utils as datetime
> > +from south.v2 import SchemaMigration
> > +
> > +
> > +class Migration(SchemaMigration):
> > +
> > + def forwards(self, orm):
> > + # Adding model 'LargeFile'
> > + db.create_table(u'maasserver_largefile', (
> > + (u'id', self.gf
> ('django.db.models.fields.AutoField')(primary_key=True)),
> > + ('created', self.gf
> ('django.db.models.fields.DateTimeField')()),
> > + ('updated', self.gf
> ('django.db.models.fields.DateTimeField')()),
> > + ('sha256', self.gf('django.db.models.fields.CharField')(unique=True,
> max_length=65)),
> > + ('total_size', self.gf
> ('django.db.models.fields.IntegerField')()),
> > + ('content', self.gf
> ('maasserver.fields.LargeObjectField')()),
> > + ))
> > + db.send_create_signal(u'maasserver', ['LargeFile'])
> > +
> > +
> > + def backwards(self, orm):
> > + # Deleting model 'LargeFile'
> > + db.delete_table(u'maasserver_largefile')
> > +
> > +
> > + models = {
> > + u'auth.group': {
> > + 'Meta': {'object_name': 'Group'},
> > + u'id': ('django.db.models.fields.AutoField', [],
> {'primary_key': 'True'}),
> > + 'name': ('django.db.models.fields.CharField', [],
> {'unique': 'True', 'max_length': '80'}),
> > + 'permissions':
> ('django.db.models.fields.related.ManyToManyField', [], {'to':
> u"orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
> > + },
> > + u'auth.permission': {
> > + 'Meta': {'ordering': "(u'content_type__app_label',
> u'content_type__model', u'codename')", 'unique_together':
> "((u'content_type', u'codename'),)", 'object_name': 'Permission'},
> > + 'codename': ('django.db.models.fields.CharField', [],
> {'max_length': '100'}),
> > + 'content_type':
> ('django.db.models.fields.related.ForeignKey', [], {'to':
> u"orm['contenttypes.ContentType']"}),
> > + u'id': ('django.db.models.fields.AutoField', [],
> {'primary_key': 'True'}),
> > + 'name': ('django.db.models.fields.CharField', [],
> {'max_length': '50'})
> > + },
> > + u'auth.user': {
> > + 'Meta': {'object_name': 'User'},
> > + 'date_joined': ('django.db.models.fields.DateTimeField',
> [], {'default': 'datetime.datetime.now'}),
> > + 'email': ('django.db.models.fields.EmailField', [],
> {'unique': 'True', 'max_length': '75', 'blank': 'Tr...

[Blake Rouse (blake-rouse)]

Jason,

Changed sha256 field to 64 length.

This command reports it as 65, 'sha256sum setup.py | awk '{print $1}' | wc'. But you are correct python does not need the '\0'.

[Jason Hobbs (jason-hobbs)]

thanks!

On Fri, Aug 8, 2014 at 2:19 PM, Blake Rouse <email address hidden>
wrote:

> Jason,
>
> Changed sha256 field to 64 length.
>
> This command reports it as 65, 'sha256sum setup.py | awk '{print $1}' |
> wc'. But you are correct python does not need the '\0'.
> --
> https://code.launchpad.net/~blake-rouse/maas/largefile-model/+merge/229949
> You are reviewing the proposed merge of
> lp:~blake-rouse/maas/largefile-model into lp:maas.
>

=== added file 'src/maasserver/migrations/0097_add_largefile_model.py'
--- src/maasserver/migrations/0097_add_largefile_model.py	1970-01-01 00:00:00 +0000
+++ src/maasserver/migrations/0097_add_largefile_model.py	2014-08-08 12:16:36 +0000
@@ -0,0 +1,344 @@
+from django.db import models
+from south.db import db
+# -*- coding: utf-8 -*-
+from south.utils import datetime_utils as datetime
+from south.v2 import SchemaMigration
+
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        # Adding model 'LargeFile'
+        db.create_table(u'maasserver_largefile', (
+            (u'id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('created', self.gf('django.db.models.fields.DateTimeField')()),
+            ('updated', self.gf('django.db.models.fields.DateTimeField')()),
+            ('sha256', self.gf('django.db.models.fields.CharField')(unique=True, max_length=64)),
+            ('total_size', self.gf('django.db.models.fields.IntegerField')()),
+            ('content', self.gf('maasserver.fields.LargeObjectField')()),
+        ))
+        db.send_create_signal(u'maasserver', ['LargeFile'])
+
+
+    def backwards(self, orm):
+        # Deleting model 'LargeFile'
+        db.delete_table(u'maasserver_largefile')
+
+
+    models = {
+        u'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        u'auth.permission': {
+            'Meta': {'ordering': "(u'content_type__app_label', u'content_type__model', u'codename')", 'unique_together': "((u'content_type', u'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['contenttypes.ContentType']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        u'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'symmetrical': 'False', 'related_name': "u'user_set'", 'blank': 'True', 'to': u"orm['auth.Group']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'symmetrical': 'False', 'related_name': "u'user_set'", 'blank': 'True', 'to': u"orm['auth.Permission']"}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        u'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        u'maasserver.bootimage': {
+            'Meta': {'unique_together': "((u'nodegroup', u'osystem', u'architecture', u'subarchitecture', u'release', u'purpose', u'label'),)", 'object_name': 'BootImage'},
+            'architecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'label': ('django.db.models.fields.CharField', [], {'default': "u'release'", 'max_length': '255'}),
+            'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
+            'osystem': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'purpose': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'release': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'subarchitecture': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'supported_subarches': ('django.db.models.fields.CharField', [], {'max_length': '255', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'xinstall_path': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'null': 'True', 'blank': 'True'}),
+            'xinstall_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '30', 'null': 'True', 'blank': 'True'})
+        },
+        u'maasserver.bootsource': {
+            'Meta': {'object_name': 'BootSource'},
+            'cluster': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'keyring_data': ('maasserver.fields.EditableBinaryField', [], {'blank': 'True'}),
+            'keyring_filename': ('django.db.models.fields.FilePathField', [], {'max_length': '100', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'url': ('django.db.models.fields.URLField', [], {'max_length': '200'})
+        },
+        u'maasserver.bootsourceselection': {
+            'Meta': {'object_name': 'BootSourceSelection'},
+            'arches': ('djorm_pgarray.fields.ArrayField', [], {'default': 'None', 'dbtype': "u'text'", 'null': 'True', 'blank': 'True'}),
+            'boot_source': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.BootSource']"}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'labels': ('djorm_pgarray.fields.ArrayField', [], {'default': 'None', 'dbtype': "u'text'", 'null': 'True', 'blank': 'True'}),
+            'release': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '20', 'blank': 'True'}),
+            'subarches': ('djorm_pgarray.fields.ArrayField', [], {'default': 'None', 'dbtype': "u'text'", 'null': 'True', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.componenterror': {
+            'Meta': {'object_name': 'ComponentError'},
+            'component': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '40'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'error': ('django.db.models.fields.CharField', [], {'max_length': '1000'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.config': {
+            'Meta': {'object_name': 'Config'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'value': ('maasserver.fields.JSONObjectField', [], {'null': 'True'})
+        },
+        u'maasserver.dhcplease': {
+            'Meta': {'object_name': 'DHCPLease'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'ip': ('maasserver.fields.MAASIPAddressField', [], {'unique': 'True', 'max_length': '39'}),
+            'mac': ('maasserver.fields.MACAddressField', [], {}),
+            'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"})
+        },
+        u'maasserver.downloadprogress': {
+            'Meta': {'object_name': 'DownloadProgress'},
+            'bytes_downloaded': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'error': ('django.db.models.fields.CharField', [], {'max_length': '1000', 'blank': 'True'}),
+            'filename': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
+            'size': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.event': {
+            'Meta': {'object_name': 'Event'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'description': ('django.db.models.fields.TextField', [], {'default': "u''", 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
+            'type': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.EventType']"}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.eventtype': {
+            'Meta': {'object_name': 'EventType'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'description': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'level': ('django.db.models.fields.IntegerField', [], {}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.filestorage': {
+            'Meta': {'unique_together': "((u'filename', u'owner'),)", 'object_name': 'FileStorage'},
+            'content': ('metadataserver.fields.BinaryField', [], {'blank': 'True'}),
+            'filename': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'key': ('django.db.models.fields.CharField', [], {'default': "u'aa7f5dec-1e3c-11e4-87da-bcee7b78dc5b'", 'unique': 'True', 'max_length': '36'}),
+            'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['auth.User']", 'null': 'True', 'blank': 'True'})
+        },
+        u'maasserver.largefile': {
+            'Meta': {'object_name': 'LargeFile'},
+            'content': ('maasserver.fields.LargeObjectField', [], {}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'sha256': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '64'}),
+            'total_size': ('django.db.models.fields.IntegerField', [], {}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.licensekey': {
+            'Meta': {'unique_together': "((u'osystem', u'distro_series'),)", 'object_name': 'LicenseKey'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'distro_series': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'license_key': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'osystem': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.macaddress': {
+            'Meta': {'ordering': "(u'created',)", 'object_name': 'MACAddress'},
+            'cluster_interface': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['maasserver.NodeGroupInterface']", 'null': 'True', 'blank': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'ip_addresses': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.StaticIPAddress']", 'symmetrical': 'False', 'through': u"orm['maasserver.MACStaticIPAddressLink']", 'blank': 'True'}),
+            'mac_address': ('maasserver.fields.MACAddressField', [], {'unique': 'True'}),
+            'networks': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Network']", 'symmetrical': 'False', 'blank': 'True'}),
+            'node': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Node']"}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.macstaticipaddresslink': {
+            'Meta': {'unique_together': "((u'ip_address', u'mac_address'),)", 'object_name': 'MACStaticIPAddressLink'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'ip_address': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.StaticIPAddress']", 'unique': 'True'}),
+            'mac_address': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.MACAddress']"}),
+            'nic_alias': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.network': {
+            'Meta': {'object_name': 'Network'},
+            'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'ip': ('maasserver.fields.MAASIPAddressField', [], {'unique': 'True', 'max_length': '39'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}),
+            'netmask': ('maasserver.fields.MAASIPAddressField', [], {'max_length': '39'}),
+            'vlan_tag': ('django.db.models.fields.PositiveSmallIntegerField', [], {'unique': 'True', 'null': 'True', 'blank': 'True'})
+        },
+        u'maasserver.node': {
+            'Meta': {'object_name': 'Node'},
+            'agent_name': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'null': 'True', 'blank': 'True'}),
+            'architecture': ('django.db.models.fields.CharField', [], {'max_length': '31'}),
+            'boot_type': ('django.db.models.fields.CharField', [], {'default': "u'fastpath'", 'max_length': '20'}),
+            'cpu_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'distro_series': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '20', 'blank': 'True'}),
+            'error': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
+            'error_description': ('django.db.models.fields.TextField', [], {'default': "u''", 'blank': 'True'}),
+            'hostname': ('django.db.models.fields.CharField', [], {'default': "u''", 'unique': 'True', 'max_length': '255', 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'license_key': ('django.db.models.fields.CharField', [], {'max_length': '30', 'null': 'True', 'blank': 'True'}),
+            'memory': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'netboot': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']", 'null': 'True'}),
+            'osystem': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '20', 'blank': 'True'}),
+            'owner': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['auth.User']", 'null': 'True', 'blank': 'True'}),
+            'power_parameters': ('maasserver.fields.JSONObjectField', [], {'default': "u''", 'blank': 'True'}),
+            'power_state': ('django.db.models.fields.CharField', [], {'default': "u'unknown'", 'max_length': '10'}),
+            'power_type': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '10', 'blank': 'True'}),
+            'routers': ('djorm_pgarray.fields.ArrayField', [], {'default': 'None', 'dbtype': "u'macaddr'", 'null': 'True', 'blank': 'True'}),
+            'status': ('django.db.models.fields.IntegerField', [], {'default': '0', 'max_length': '10'}),
+            'storage': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'system_id': ('django.db.models.fields.CharField', [], {'default': "u'node-aa7fb684-1e3c-11e4-87da-bcee7b78dc5b'", 'unique': 'True', 'max_length': '41'}),
+            'tags': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['maasserver.Tag']", 'symmetrical': 'False'}),
+            'token': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['piston.Token']", 'null': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'zone': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.Zone']", 'on_delete': 'models.SET_DEFAULT'})
+        },
+        u'maasserver.nodegroup': {
+            'Meta': {'object_name': 'NodeGroup'},
+            'api_key': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '18'}),
+            'api_token': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['piston.Token']", 'unique': 'True'}),
+            'cluster_name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '100', 'blank': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'dhcp_key': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'maas_url': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
+            'status': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'uuid': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'})
+        },
+        u'maasserver.nodegroupinterface': {
+            'Meta': {'unique_together': "((u'nodegroup', u'name'),)", 'object_name': 'NodeGroupInterface'},
+            'broadcast_ip': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'foreign_dhcp_ip': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'interface': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
+            'ip': ('maasserver.fields.MAASIPAddressField', [], {'max_length': '39'}),
+            'ip_range_high': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'ip_range_low': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'management': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'name': ('django.db.models.fields.CharField', [], {'default': "u''", 'max_length': '255', 'blank': 'True'}),
+            'nodegroup': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['maasserver.NodeGroup']"}),
+            'router_ip': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'static_ip_range_high': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'static_ip_range_low': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'subnet_mask': ('maasserver.fields.MAASIPAddressField', [], {'default': 'None', 'max_length': '39', 'null': 'True', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.sshkey': {
+            'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSHKey'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'key': ('django.db.models.fields.TextField', [], {}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['auth.User']"})
+        },
+        u'maasserver.sslkey': {
+            'Meta': {'unique_together': "((u'user', u'key'),)", 'object_name': 'SSLKey'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'key': ('django.db.models.fields.TextField', [], {}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['auth.User']"})
+        },
+        u'maasserver.staticipaddress': {
+            'Meta': {'object_name': 'StaticIPAddress'},
+            'alloc_type': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'ip': ('maasserver.fields.MAASIPAddressField', [], {'unique': 'True', 'max_length': '39'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['auth.User']", 'null': 'True', 'blank': 'True'})
+        },
+        u'maasserver.tag': {
+            'Meta': {'object_name': 'Tag'},
+            'comment': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'definition': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'kernel_opts': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'maasserver.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': u"orm['auth.User']", 'unique': 'True'})
+        },
+        u'maasserver.zone': {
+            'Meta': {'ordering': "[u'name']", 'object_name': 'Zone'},
+            'created': ('django.db.models.fields.DateTimeField', [], {}),
+            'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '256'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'piston.consumer': {
+            'Meta': {'object_name': 'Consumer'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
+            'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '16'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'consumers'", 'null': 'True', 'to': u"orm['auth.User']"})
+        },
+        u'piston.token': {
+            'Meta': {'object_name': 'Token'},
+            'callback': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
+            'callback_confirmed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'consumer': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['piston.Consumer']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_approved': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'key': ('django.db.models.fields.CharField', [], {'max_length': '18'}),
+            'secret': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
+            'timestamp': ('django.db.models.fields.IntegerField', [], {'default': '1407420663L'}),
+            'token_type': ('django.db.models.fields.IntegerField', [], {}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'tokens'", 'null': 'True', 'to': u"orm['auth.User']"}),
+            'verifier': ('django.db.models.fields.CharField', [], {'max_length': '10'})
+        }
+    }
+
+    complete_apps = ['maasserver']

=== modified file 'src/maasserver/models/__init__.py'
--- src/maasserver/models/__init__.py	2014-07-18 15:45:22 +0000
+++ src/maasserver/models/__init__.py	2014-08-08 12:16:36 +0000
@@ -22,6 +22,7 @@
     'DownloadProgress',
     'Event',
     'FileStorage',
+    'LargeFile',
     'LicenseKey',
     'logger',
     'MACAddress',
@@ -56,6 +57,7 @@
 from maasserver.models.event import Event
 from maasserver.models.eventtype import EventType
 from maasserver.models.filestorage import FileStorage
+from maasserver.models.largefile import LargeFile
 from maasserver.models.licensekey import LicenseKey
 from maasserver.models.macaddress import MACAddress
 from maasserver.models.macipaddresslink import MACStaticIPAddressLink
@@ -77,9 +79,9 @@
 # export in __all__.
 ignore_unused(
     BootImage, ComponentError, Config, DHCPLease, DownloadProgress,
-    Event, EventType, FileStorage, LicenseKey, StaticIPAddress, MACAddress,
-    MACStaticIPAddressLink, Network, NodeGroup, SSHKey, Tag, UserProfile,
-    NodeGroupInterface, Zone, logger)
+    Event, EventType, FileStorage, LargeFile, LicenseKey, StaticIPAddress,
+    MACAddress, MACStaticIPAddressLink, Network, NodeGroup, SSHKey, Tag,
+    UserProfile, NodeGroupInterface, Zone, logger)
 
 
 # Connect the 'create_user' method to the post save signal of User.

=== added file 'src/maasserver/models/largefile.py'
--- src/maasserver/models/largefile.py	1970-01-01 00:00:00 +0000
+++ src/maasserver/models/largefile.py	2014-08-08 12:16:36 +0000
@@ -0,0 +1,108 @@
+# Copyright 2014 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Large file storage."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+str = None
+
+__metaclass__ = type
+__all__ = [
+    'LargeFile',
+    ]
+
+import os
+
+from django.db.models import (
+    BigIntegerField,
+    CharField,
+    Manager,
+    )
+from django.db.models.signals import post_delete
+from django.dispatch import receiver
+from maasserver import DefaultMeta
+from maasserver.fields import LargeObjectField
+from maasserver.models.cleansave import CleanSave
+from maasserver.models.timestampedmodel import TimestampedModel
+from maasserver.utils.orm import get_one
+
+
+class FileStorageManager(Manager):
+    """Manager for `LargeFile` objects."""
+
+    def has_file(self, sha256):
+        """True if file with sha256 value exists."""
+        return self.filter(sha256=sha256).exists()
+
+    def get_file(self, sha256):
+        """Return file based on SHA256 value."""
+        return get_one(self.filter(sha256=sha256))
+
+
+class LargeFile(CleanSave, TimestampedModel):
+    """Files that are stored in the large object storage.
+
+    Only unique files are stored in the database, as only one sha256 value
+    can exist per file. This provides data deduplication on the file level.
+
+    Currently only used by `BootResourceFile`. This speeds up the import
+    process by only saving unique files.
+
+    :ivar sha256: Calculated SHA256 value of `content`.
+    :ivar total_size: Final size of `content`. The data might currently
+        be saving, so total_size could be larger than `size`. `size` should
+        never be larger than `total_size`.
+    :ivar content: File data.
+    """
+
+    class Meta(DefaultMeta):
+        """Needed for South to recognize this model."""
+
+    objects = FileStorageManager()
+
+    sha256 = CharField(max_length=64, unique=True, editable=False)
+
+    total_size = BigIntegerField(editable=False)
+
+    # content is stored directly in the database, in the large object storage.
+    # Max file storage size is 4TB.
+    content = LargeObjectField()
+
+    @property
+    def size(self):
+        """Size of content."""
+        with self.content.open('rb') as stream:
+            stream.seek(0, os.SEEK_END)
+            size = stream.tell()
+        return size
+
+    @property
+    def progress(self):
+        """Precentage of `content` saved."""
+        if self.size <= 0:
+            # Handle division of zero
+            return 0
+        return self.total_size / float(self.size)
+
+    @property
+    def complete(self):
+        """`content` has been completely saved."""
+        return (self.total_size == self.size)
+
+
+@receiver(post_delete)
+def delete_large_object(sender, instance, **kwargs):
+    """Delete the large object when the `LargeFile` is deleted.
+
+    This is done using the `post_delete` signal instead of overriding delete
+    on `LargeFile`, so it works correctly for both the model and
+    `QuerySet`.
+    """
+    if sender == LargeFile:
+        if instance.content is not None:
+            instance.content.unlink()

=== added file 'src/maasserver/models/tests/test_largefile.py'
--- src/maasserver/models/tests/test_largefile.py	1970-01-01 00:00:00 +0000
+++ src/maasserver/models/tests/test_largefile.py	2014-08-08 12:16:36 +0000
@@ -0,0 +1,92 @@
+# Copyright 2014 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for :class:`LargeFile`."""
+
+from __future__ import (
+    absolute_import,
+    print_function,
+    unicode_literals,
+    )
+
+str = None
+
+__metaclass__ = type
+__all__ = []
+
+from random import randint
+
+from maasserver.models.largefile import LargeFile
+from maasserver.testing.factory import factory
+from maasserver.testing.testcase import MAASServerTestCase
+from maastesting.matchers import MockCalledOnceWith
+
+
+class TestLargeFileManager(MAASServerTestCase):
+
+    def test_has_file(self):
+        largefile = factory.make_large_file()
+        self.assertTrue(LargeFile.objects.has_file(largefile.sha256))
+
+    def test_get_file(self):
+        largefile = factory.make_large_file()
+        obj = LargeFile.objects.get_file(largefile.sha256)
+        self.assertEqual(largefile, obj)
+
+
+class TestLargeFile(MAASServerTestCase):
+
+    def test_content(self):
+        size = randint(512, 1024)
+        content = factory.make_string(size=size)
+        largefile = factory.make_large_file(content, size=size)
+        with largefile.content.open('rb') as stream:
+            data = stream.read()
+        self.assertEqual(content, data)
+
+    def test_empty_content(self):
+        size = 0
+        content = ""
+        largefile = factory.make_large_file(content, size=size)
+        with largefile.content.open('rb') as stream:
+            data = stream.read()
+        self.assertEqual(content, data)
+
+    def test_size(self):
+        size = randint(512, 1024)
+        total_size = randint(1025, 2048)
+        content = factory.make_string(size=size)
+        largefile = factory.make_large_file(content, size=total_size)
+        self.assertEqual(size, largefile.size)
+
+    def test_progress(self):
+        size = randint(512, 1024)
+        total_size = randint(1025, 2048)
+        content = factory.make_string(size=size)
+        largefile = factory.make_large_file(content, size=total_size)
+        self.assertEqual(total_size / float(size), largefile.progress)
+
+    def test_progress_of_empty_file(self):
+        size = 0
+        content = ""
+        largefile = factory.make_large_file(content, size=size)
+        self.assertEqual(0, largefile.progress)
+
+    def test_complete_returns_False_when_content_incomplete(self):
+        size = randint(512, 1024)
+        total_size = randint(1025, 2048)
+        content = factory.make_string(size=size)
+        largefile = factory.make_large_file(content, size=total_size)
+        self.assertFalse(largefile.complete)
+
+    def test_complete_returns_True_when_content_is_complete(self):
+        largefile = factory.make_large_file()
+        self.assertTrue(largefile.complete)
+
+    def test_delete_calls_unlink_on_content(self):
+        largefile = factory.make_large_file()
+        content = largefile.content
+        self.addCleanup(content.unlink)
+        unlink_mock = self.patch(content, 'unlink')
+        largefile.delete()
+        self.assertThat(unlink_mock, MockCalledOnceWith())

=== modified file 'src/maasserver/testing/factory.py'
--- src/maasserver/testing/factory.py	2014-08-01 11:19:44 +0000
+++ src/maasserver/testing/factory.py	2014-08-08 12:16:36 +0000
@@ -17,6 +17,7 @@
     "Messages",
     ]
 
+import hashlib
 from io import BytesIO
 import logging
 import random
@@ -32,7 +33,10 @@
     NODEGROUPINTERFACE_MANAGEMENT,
     POWER_STATE,
     )
-from maasserver.fields import MAC
+from maasserver.fields import (
+    LargeObjectFile,
+    MAC,
+    )
 from maasserver.models import (
     BootImage,
     BootSource,
@@ -42,6 +46,7 @@
     Event,
     EventType,
     FileStorage,
+    LargeFile,
     LicenseKey,
     MACAddress,
     MACStaticIPAddressLink,
@@ -950,6 +955,26 @@
             type = self.make_event_type()
         return Event.objects.create(node=node, type=type)
 
+    def make_large_file(self, content=None, size=512):
+        """Create `LargeFile`.
+
+        :param content: Data to store in large file object.
+        :param size: Size of `content`. If `content` is None
+            then it will be a random string of this size. If content is
+            provided and `size` is not the same length, then it will
+            be an inprogress file.
+        """
+        if content is None:
+            content = factory.make_string(size=size)
+        sha256 = hashlib.sha256()
+        sha256.update(content)
+        sha256 = sha256.hexdigest()
+        largeobject = LargeObjectFile()
+        with largeobject.open('wb') as stream:
+            stream.write(content)
+        return LargeFile.objects.create(
+            sha256=sha256, total_size=size, content=largeobject)
+
 
 # Create factory singleton.
 factory = Factory()