MAAS

Merge lp:~blake-rouse/maas/add-raid-and-bcache-types into lp:~maas-committers/maas/trunk

  1. add-raid-and-bcache-types
  2. Merge into trunk
Proposed by Blake Rouse
Status: Merged
Approved by: Blake Rouse
Approved revision: no longer in the source branch.
Merged at revision: 4060
Proposed branch: lp:~blake-rouse/maas/add-raid-and-bcache-types
Merge into: lp:~maas-committers/maas/trunk
Diff against target: 1322 lines (+973/-49)
12 files modified
src/maasserver/api/tests/test_blockdevice.py (+2/-1)
src/maasserver/api/tests/test_events.py (+2/-2)
src/maasserver/enum.py (+49/-0)
src/maasserver/models/blockdevice.py (+6/-2)
src/maasserver/models/filesystemgroup.py (+214/-6)
src/maasserver/models/tests/test_filesystemgroup.py (+579/-13)
src/maasserver/models/tests/test_virtualblockdevice.py (+22/-0)
src/maasserver/models/virtualblockdevice.py (+7/-4)
src/maasserver/testing/factory.py (+76/-12)
src/maasserver/tests/test_forms_blockdevice.py (+7/-2)
src/maasserver/utils/version.py (+1/-1)
src/provisioningserver/tests/test_service_monitor.py (+8/-6)
To merge this branch: bzr merge lp:~blake-rouse/maas/add-raid-and-bcache-types
Reviewer Review Type Date Requested Status
Raphaël Badin (community) Approve
Review via email: mp+263405@code.launchpad.net

Commit message

Add RAID and Bcache to the FilesystemGroup model. Drive-by fix for lint issues that showed up once an updated linter was installed from as system update.

This adds the filesystem types and filesystem group types for both RAID and Bcache. Validation is performed to make sure that the filesystems added to the filesystem group is correct based on the type.

To post a comment you must log in.
Revision history for this message
Blake Rouse (blake-rouse) wrote :

Sorry about this size, but I think you will appreciate it. Its mostly the tests that add a lot to this MP, since every combination is tested.

Revision history for this message
Raphaël Badin (rvb) wrote :

Nice! I've got a bunch of comments but nothing major.

> Sorry about this size, but I think you will appreciate it.\

I do ;). Thanks for the extensive testing.

review: Approve
Revision history for this message
Blake Rouse (blake-rouse) wrote :

Thanks for the review.

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (56.3 KiB)

The attempt to merge lp:~blake-rouse/maas/add-raid-and-bcache-types into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Get:1 http://security.ubuntu.com trusty-security Release.gpg [933 B]
Get:2 http://security.ubuntu.com trusty-security Release [63.5 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates Release [63.5 kB]
Get:5 http://security.ubuntu.com trusty-security/main Sources [87.4 kB]
Get:6 http://security.ubuntu.com trusty-security/universe Sources [27.8 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Get:7 http://security.ubuntu.com trusty-security/main amd64 Packages [304 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Get:8 http://security.ubuntu.com trusty-security/universe amd64 Packages [110 kB]
Get:9 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [212 kB]
Hit http://security.ubuntu.com trusty-security/main Translation-en
Get:10 http://security.ubuntu.com trusty-security/universe Translation-en [63.5 kB]
Get:11 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [123 kB]
Get:12 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [562 kB]
Get:13 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [292 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Get:14 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en [154 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 2,064 kB in 3s (672 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-iscpy python-jinja2 python-jsonschema python-lockfile python-lxml python-mock p...

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (81.9 KiB)

The attempt to merge lp:~blake-rouse/maas/add-raid-and-bcache-types into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Get:1 http://security.ubuntu.com trusty-security Release.gpg [933 B]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Get:2 http://security.ubuntu.com trusty-security Release [63.5 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates Release [63.5 kB]
Get:5 http://security.ubuntu.com trusty-security/main Sources [87.4 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Get:6 http://security.ubuntu.com trusty-security/universe Sources [27.8 kB]
Get:7 http://security.ubuntu.com trusty-security/main amd64 Packages [304 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Get:8 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [212 kB]
Get:9 http://security.ubuntu.com trusty-security/universe amd64 Packages [110 kB]
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Get:10 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [123 kB]
Get:11 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [562 kB]
Get:12 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [292 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 1,847 kB in 3s (605 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-iscpy python-jinja2 python-jsonschema python-lockfile python-lxml python-mock python-netaddr python-neti...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/maasserver/api/tests/test_blockdevice.py'
2--- src/maasserver/api/tests/test_blockdevice.py 2015-07-01 14:14:14 +0000
3+++ src/maasserver/api/tests/test_blockdevice.py 2015-07-01 21:29:38 +0000
4@@ -63,6 +63,7 @@
5 # Make a filesystem_group (analogous to a volume group) on top of our
6 # two lvm-pm filesystems.
7 filesystem_group = factory.make_FilesystemGroup(
8+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
9 filesystems=lvm_pv_filesystems)
10
11 # Make a VirtualBlockDevice on top of the filesystem group we just
12@@ -715,7 +716,7 @@
13 httplib.FORBIDDEN, response.status_code, response.content)
14
15 def test_update_virtual_block_device_as_normal_user(self):
16- """Check update block device with a virtual one fails for normal user.
17+ """Check update block device with a virtual one.
18 """
19 node = factory.make_Node(owner=self.logged_in_user)
20 block_device = factory.make_VirtualBlockDevice(node=node,
21
22=== modified file 'src/maasserver/api/tests/test_events.py'
23--- src/maasserver/api/tests/test_events.py 2015-04-07 10:22:09 +0000
24+++ src/maasserver/api/tests/test_events.py 2015-07-01 21:29:38 +0000
25@@ -590,10 +590,10 @@
26 # Calculate the expected values for limit
27 # and start event id
28 limit = events_module.DEFAULT_EVENT_LOG_LIMIT \
29- if not 'limit' in expected_params \
30+ if 'limit' not in expected_params \
31 else int(expected_params['limit'])
32
33- start_id = 0 if not 'after' in expected_params \
34+ start_id = 0 if 'after' not in expected_params \
35 else int(expected_params['after'])
36
37 expected_params['after'] = unicode(start_id + limit)
38
39=== modified file 'src/maasserver/enum.py'
40--- src/maasserver/enum.py 2015-06-29 08:53:42 +0000
41+++ src/maasserver/enum.py 2015-07-01 21:29:38 +0000
42@@ -361,6 +361,18 @@
43 #: LVM Physical Volume.
44 LVM_PV = 'lvm-pv'
45
46+ #: RAID.
47+ RAID = 'raid'
48+
49+ #: RAID spare.
50+ RAID_SPARE = 'raid-spare'
51+
52+ #: Bcache cache.
53+ BCACHE_CACHE = 'bcache-cache'
54+
55+ #: Bcache backing.
56+ BCACHE_BACKING = 'bcache-backing'
57+
58
59 # Django choices for FILESYSTEM_TYPE: sequence of tuples (key, UI
60 # representation).
61@@ -368,6 +380,10 @@
62 (FILESYSTEM_TYPE.EXT3, "ext3"),
63 (FILESYSTEM_TYPE.EXT4, "ext4"),
64 (FILESYSTEM_TYPE.LVM_PV, "lvm"),
65+ (FILESYSTEM_TYPE.RAID, "raid"),
66+ (FILESYSTEM_TYPE.RAID_SPARE, "raid-spare"),
67+ (FILESYSTEM_TYPE.BCACHE_CACHE, "bcache-cache"),
68+ (FILESYSTEM_TYPE.BCACHE_BACKING, "bcache-backing"),
69 )
70
71
72@@ -376,11 +392,44 @@
73 #: LVM volume group.
74 LVM_VG = 'lvm-vg'
75
76+ #: RAID level 0
77+ RAID_0 = 'raid-0'
78+
79+ #: RAID level 1
80+ RAID_1 = 'raid-1'
81+
82+ #: RAID level 4
83+ RAID_4 = 'raid-4'
84+
85+ #: RAID level 5
86+ RAID_5 = 'raid-5'
87+
88+ #: RAID level 6
89+ RAID_6 = 'raid-6'
90+
91+ #: Bcache
92+ BCACHE = 'bcache'
93+
94+
95+FILESYSTEM_GROUP_RAID_TYPES = [
96+ FILESYSTEM_GROUP_TYPE.RAID_0,
97+ FILESYSTEM_GROUP_TYPE.RAID_1,
98+ FILESYSTEM_GROUP_TYPE.RAID_4,
99+ FILESYSTEM_GROUP_TYPE.RAID_5,
100+ FILESYSTEM_GROUP_TYPE.RAID_6,
101+ ]
102+
103
104 # Django choices for FILESYSTEM_GROUP_TYPE: sequence of tuples (key, UI
105 # representation).
106 FILESYSTEM_GROUP_TYPE_CHOICES = (
107 (FILESYSTEM_GROUP_TYPE.LVM_VG, "LVM VG"),
108+ (FILESYSTEM_GROUP_TYPE.RAID_0, "RAID 0"),
109+ (FILESYSTEM_GROUP_TYPE.RAID_1, "RAID 1"),
110+ (FILESYSTEM_GROUP_TYPE.RAID_4, "RAID 4"),
111+ (FILESYSTEM_GROUP_TYPE.RAID_5, "RAID 5"),
112+ (FILESYSTEM_GROUP_TYPE.RAID_6, "RAID 6"),
113+ (FILESYSTEM_GROUP_TYPE.BCACHE, "Bcache"),
114 )
115
116
117
118=== modified file 'src/maasserver/models/blockdevice.py'
119--- src/maasserver/models/blockdevice.py 2015-06-27 01:49:24 +0000
120+++ src/maasserver/models/blockdevice.py 2015-07-01 21:29:38 +0000
121@@ -37,6 +37,10 @@
122 from maasserver.utils.orm import psql_array
123
124
125+MIN_BLOCK_DEVICE_SIZE = 143360 # The size of an Apple II disk
126+MIN_BLOCK_DEVICE_BLOCK_SIZE = 512 # A ProDOS block
127+
128+
129 class BlockDeviceManager(Manager):
130 """Manager for `BlockDevice` class."""
131
132@@ -108,12 +112,12 @@
133
134 size = BigIntegerField(
135 blank=False, null=False,
136- validators=[MinValueValidator(143360)], # The size of an Apple II disk
137+ validators=[MinValueValidator(MIN_BLOCK_DEVICE_SIZE)],
138 help_text="Size of block device in bytes.")
139
140 block_size = IntegerField(
141 blank=False, null=False,
142- validators=[MinValueValidator(512)], # A ProDOS block
143+ validators=[MinValueValidator(MIN_BLOCK_DEVICE_BLOCK_SIZE)],
144 help_text="Size of a block on the device in bytes.")
145
146 tags = ArrayField(
147
148=== modified file 'src/maasserver/models/filesystemgroup.py'
149--- src/maasserver/models/filesystemgroup.py 2015-06-30 15:30:19 +0000
150+++ src/maasserver/models/filesystemgroup.py 2015-07-01 21:29:38 +0000
151@@ -17,12 +17,18 @@
152 'FilesystemGroup',
153 ]
154
155+from collections import Counter
156 from uuid import uuid4
157
158 from django.core.exceptions import ValidationError
159 from django.db.models import CharField
160 from maasserver import DefaultMeta
161-from maasserver.enum import FILESYSTEM_GROUP_TYPE_CHOICES
162+from maasserver.enum import (
163+ FILESYSTEM_GROUP_RAID_TYPES,
164+ FILESYSTEM_GROUP_TYPE,
165+ FILESYSTEM_GROUP_TYPE_CHOICES,
166+ FILESYSTEM_TYPE,
167+)
168 from maasserver.models.cleansave import CleanSave
169 from maasserver.models.timestampedmodel import TimestampedModel
170
171@@ -44,7 +50,8 @@
172 max_length=36, unique=True, null=False, blank=False, editable=False)
173
174 group_type = CharField(
175- max_length=20, choices=FILESYSTEM_GROUP_TYPE_CHOICES)
176+ max_length=20, null=False, blank=False,
177+ choices=FILESYSTEM_GROUP_TYPE_CHOICES)
178
179 name = CharField(
180 max_length=255, null=False, blank=False)
181@@ -66,11 +73,93 @@
182 `VirtualBlockDevice` should calculate its size from this filesystem
183 group.
184 """
185+ if self.is_lvm():
186+ return self.get_lvm_size()
187+ elif self.is_raid():
188+ return self.get_raid_size()
189+ elif self.is_bcache():
190+ return self.get_bcache_size()
191+ else:
192+ return 0
193+
194+ def get_lvm_size(self):
195+ """Size of this LVM volume group.
196+
197+ Calculated from the total size of all filesystems in this group.
198+ Its not calculated from its virtual_block_device size.
199+
200+ Note: Should only be called when the `group_type` is LVM_VG.
201+ """
202 return sum(
203 filesystem.get_size()
204 for filesystem in self.filesystems.all()
205 )
206
207+ def get_smallest_filesystem_size(self):
208+ """Return the smallest filesystem size."""
209+ filesystems = list(self.filesystems.all())
210+ if len(filesystems) == 0:
211+ return 0
212+ else:
213+ return min(
214+ filesystem.get_size()
215+ for filesystem in filesystems
216+ )
217+
218+ def get_raid_size(self):
219+ """Size of this RAID.
220+
221+ Calculated based on the RAID type and how output size based on that
222+ type. The size will be calculated using the smallest size filesystem
223+ attached to this RAID. The linked `VirtualBlockDevice` should
224+ calculate its size from this filesystem group.
225+
226+ Note: Should only be called when the `group_type` is in
227+ `FILESYSTEM_GROUP_RAID_TYPES`.
228+ """
229+ min_size = self.get_smallest_filesystem_size()
230+ if min_size <= 0:
231+ # Possible when no filesytems are attached to this group.
232+ return 0
233+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_0:
234+ return min_size
235+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_1:
236+ return min_size
237+ else:
238+ num_raid = len([
239+ fstype
240+ for fstype in self._get_all_fstypes()
241+ if fstype == FILESYSTEM_TYPE.RAID
242+ ])
243+ if (self.group_type == FILESYSTEM_GROUP_TYPE.RAID_4 or
244+ self.group_type == FILESYSTEM_GROUP_TYPE.RAID_5):
245+ return min_size * (num_raid - 1)
246+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_6:
247+ return min_size * (num_raid - 2)
248+ raise ValueError("Unknown raid type: %s" % self.group_type)
249+
250+ def get_bcache_backing_filesystem(self):
251+ """Return the filesystem that is the backing device for the Bcache."""
252+ for filesystem in self.filesystems.all():
253+ if filesystem.fstype == FILESYSTEM_TYPE.BCACHE_BACKING:
254+ return filesystem
255+ return None
256+
257+ def get_bcache_size(self):
258+ """Size of this Bcache.
259+
260+ Calculated based on the size of the backing device. The linked
261+ `VirtualBlockDevice` should calculate its size from this
262+ filesystem group.
263+
264+ Note: Should only be called when the `group_type` is BCACHE.
265+ """
266+ backing_filesystem = self.get_bcache_backing_filesystem()
267+ if backing_filesystem is None:
268+ return 0
269+ else:
270+ return backing_filesystem.get_size()
271+
272 def get_lvm_allocated_size(self):
273 """Returns the space already allocated to virtual block devices.
274
275@@ -84,32 +173,151 @@
276
277 def get_lvm_free_space(self):
278 """Returns the total unallocated space on this FilesystemGroup"""
279- return self.get_size() - self.get_lvm_allocated_size()
280+ return self.get_lvm_size() - self.get_lvm_allocated_size()
281
282 def clean(self, *args, **kwargs):
283 super(FilesystemGroup, self).clean(*args, **kwargs)
284
285- # We allow the initial save to not have filesystems linked, any
286+ # We allow the initial save to skip model validation, any
287 # additional saves required filesystems linked. This is because the
288 # object needs to exist in the database before the filesystems can
289 # be linked.
290 if not self.id:
291 return
292
293+ # Grab all filesystems so that if the filesystems have been precached
294+ # it will be used. This prevents extra database queries.
295+ filesystems = list(self.filesystems.all())
296+
297 # Must at least have a filesystem added to the group.
298- if self.filesystems.count() == 0:
299+ if len(filesystems) == 0:
300 raise ValidationError(
301 "At least one filesystem must have been added.")
302
303 # All filesystems must belong all to the same node.
304 nodes = {
305 filesystem.get_node()
306- for filesystem in self.filesystems.all()
307+ for filesystem in filesystems
308 }
309 if len(nodes) > 1:
310 raise ValidationError(
311 "All added filesystems must belong to the same node.")
312
313+ # Validate all the different group types.
314+ if self.is_lvm():
315+ self._validate_lvm(filesystems=filesystems)
316+ elif self.is_raid():
317+ self._validate_raid(filesystems=filesystems)
318+ elif self.is_bcache():
319+ self._validate_bcache(filesystems=filesystems)
320+
321+ def is_lvm(self):
322+ """Return True if `group_type` is LVM_VG type."""
323+ return self.group_type == FILESYSTEM_GROUP_TYPE.LVM_VG
324+
325+ def is_raid(self):
326+ """Return True if `group_type` is a RAID type."""
327+ return self.group_type in FILESYSTEM_GROUP_RAID_TYPES
328+
329+ def is_bcache(self):
330+ """Return True if `group_type` is BCACHE type."""
331+ return self.group_type == FILESYSTEM_GROUP_TYPE.BCACHE
332+
333+ def _get_all_fstypes(self, filesystems=None):
334+ """Return list of all filesystem types attached."""
335+ # Grab all filesystems so that if the filesystems have been
336+ # precached it will be used. This prevents extra database queries.
337+ if filesystems is None:
338+ filesystems = list(self.filesystems.all())
339+ return [
340+ filesystem.fstype
341+ for filesystem in filesystems
342+ ]
343+
344+ def _validate_lvm(self, filesystems=None):
345+ """Validate attached filesystems are correct type for LVM_VG.
346+ """
347+ if not self.is_lvm():
348+ return
349+ unique_fstypes = set(self._get_all_fstypes(filesystems=filesystems))
350+ if unique_fstypes != set([FILESYSTEM_TYPE.LVM_PV]):
351+ raise ValidationError(
352+ "Volume group can only contain lvm physical volumes.")
353+
354+ def _validate_raid(self, filesystems=None):
355+ """Validate attached filesystems are correct count and type for RAID.
356+ """
357+ if not self.is_raid():
358+ return
359+ fstypes = self._get_all_fstypes(filesystems=filesystems)
360+ num_raid = len([
361+ fstype
362+ for fstype in fstypes
363+ if fstype == FILESYSTEM_TYPE.RAID
364+ ])
365+ num_raid_spare = len([
366+ fstype
367+ for fstype in fstypes
368+ if fstype == FILESYSTEM_TYPE.RAID_SPARE
369+ ])
370+ if self.group_type == FILESYSTEM_GROUP_TYPE.RAID_0:
371+ # RAID 0 can only contain 2 RAID filesystems and no spares are
372+ # allowed.
373+ if num_raid != 2 or num_raid_spare != 0:
374+ raise ValidationError(
375+ "RAID level 0 must have exactly 2 raid devices and "
376+ "no spares.")
377+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_1:
378+ # RAID 1 must have at least 2 RAID filesystems and should not
379+ # have any spares.
380+ if num_raid < 2 or num_raid_spare != 0:
381+ raise ValidationError(
382+ "RAID level 1 must have atleast 2 raid devices and "
383+ "no spares.")
384+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_4:
385+ # RAID 4 must have at least 3 RAID filesystems, but can have
386+ # spares.
387+ if num_raid < 3:
388+ raise ValidationError(
389+ "RAID level 4 must have atleast 3 raid devices and "
390+ "any number of spares.")
391+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_5:
392+ # RAID 5 must have at least 3 RAID filesystems, but can have
393+ # spares.
394+ if num_raid < 3:
395+ raise ValidationError(
396+ "RAID level 5 must have atleast 3 raid devices and "
397+ "any number of spares.")
398+ elif self.group_type == FILESYSTEM_GROUP_TYPE.RAID_6:
399+ # RAID 6 must have at least 4 RAID filesystems, but can have
400+ # spares.
401+ if num_raid < 4:
402+ raise ValidationError(
403+ "RAID level 6 must have atleast 4 raid devices and "
404+ "any number of spares.")
405+ num_raid_invalid = len([
406+ fstype
407+ for fstype in fstypes
408+ if (fstype != FILESYSTEM_TYPE.RAID and
409+ fstype != FILESYSTEM_TYPE.RAID_SPARE)
410+ ])
411+ if num_raid_invalid > 0:
412+ raise ValidationError(
413+ "RAID can only contain raid device and raid spares.")
414+
415+ def _validate_bcache(self, filesystems=None):
416+ """Validate attached filesystems are correct type for BCACHE.
417+ """
418+ if not self.is_bcache():
419+ return
420+ fstypes_counter = Counter(
421+ self._get_all_fstypes(filesystems=filesystems))
422+ valid_counter = Counter(
423+ [FILESYSTEM_TYPE.BCACHE_CACHE, FILESYSTEM_TYPE.BCACHE_BACKING])
424+ if fstypes_counter != valid_counter:
425+ raise ValidationError(
426+ "Bcache must contain one cache and one backing device.")
427+
428 def save(self, *args, **kwargs):
429 if not self.uuid:
430 self.uuid = uuid4()
431
432=== modified file 'src/maasserver/models/tests/test_filesystemgroup.py'
433--- src/maasserver/models/tests/test_filesystemgroup.py 2015-06-30 15:30:19 +0000
434+++ src/maasserver/models/tests/test_filesystemgroup.py 2015-07-01 21:29:38 +0000
435@@ -20,9 +20,11 @@
436
437 from django.core.exceptions import ValidationError
438 from maasserver.enum import (
439+ FILESYSTEM_GROUP_RAID_TYPES,
440 FILESYSTEM_GROUP_TYPE,
441 FILESYSTEM_TYPE,
442 )
443+from maasserver.models.blockdevice import MIN_BLOCK_DEVICE_SIZE
444 from maasserver.models.filesystemgroup import FilesystemGroup
445 from maasserver.testing.factory import factory
446 from maasserver.testing.testcase import MAASServerTestCase
447@@ -47,23 +49,220 @@
448 fsgroup = FilesystemGroup()
449 self.assertIsNone(fsgroup.get_node())
450
451- def test_get_size_returns_sum_of_all_filesystem_sizes(self):
452+ def test_get_size_returns_0_if_lvm_without_filesystems(self):
453+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.LVM_VG)
454+ self.assertEquals(0, fsgroup.get_size())
455+
456+ def test_get_size_returns_sum_of_all_filesystem_sizes_for_lvm(self):
457 node = factory.make_Node()
458 total_size = 0
459 filesystems = []
460 for _ in range(3):
461- size = random.randint(143360, 10 ** 6)
462+ size = random.randint(
463+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
464 total_size += size
465 block_device = factory.make_PhysicalBlockDevice(
466 node=node, size=size)
467 filesystems.append(
468- factory.make_Filesystem(block_device=block_device))
469- fsgroup = factory.make_FilesystemGroup(filesystems=filesystems)
470+ factory.make_Filesystem(
471+ fstype=FILESYSTEM_TYPE.LVM_PV, block_device=block_device))
472+ fsgroup = factory.make_FilesystemGroup(
473+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG, filesystems=filesystems)
474 self.assertEquals(total_size, fsgroup.get_size())
475
476- def test_get_size_returns_0_if_no_filesystems(self):
477+ def test_get_size_returns_0_if_raid_without_filesystems(self):
478+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.RAID_0)
479+ self.assertEquals(0, fsgroup.get_size())
480+
481+ def test_get_size_returns_smallest_disk_size_for_raid_0(self):
482+ node = factory.make_Node()
483+ small_size = random.randint(
484+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
485+ large_size = random.randint(small_size + 1, small_size + (10 ** 5))
486+ filesystems = [
487+ factory.make_Filesystem(
488+ fstype=FILESYSTEM_TYPE.RAID,
489+ block_device=factory.make_PhysicalBlockDevice(
490+ node=node, size=small_size)),
491+ factory.make_Filesystem(
492+ fstype=FILESYSTEM_TYPE.RAID,
493+ block_device=factory.make_PhysicalBlockDevice(
494+ node=node, size=large_size)),
495+ ]
496+ fsgroup = factory.make_FilesystemGroup(
497+ group_type=FILESYSTEM_GROUP_TYPE.RAID_0, filesystems=filesystems)
498+ self.assertEquals(small_size, fsgroup.get_size())
499+
500+ def test_get_size_returns_smallest_disk_size_for_raid_1(self):
501+ node = factory.make_Node()
502+ small_size = random.randint(
503+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
504+ large_size = random.randint(small_size + 1, small_size + (10 ** 5))
505+ filesystems = [
506+ factory.make_Filesystem(
507+ fstype=FILESYSTEM_TYPE.RAID,
508+ block_device=factory.make_PhysicalBlockDevice(
509+ node=node, size=small_size)),
510+ factory.make_Filesystem(
511+ fstype=FILESYSTEM_TYPE.RAID,
512+ block_device=factory.make_PhysicalBlockDevice(
513+ node=node, size=large_size)),
514+ ]
515+ fsgroup = factory.make_FilesystemGroup(
516+ group_type=FILESYSTEM_GROUP_TYPE.RAID_1, filesystems=filesystems)
517+ self.assertEquals(small_size, fsgroup.get_size())
518+
519+ def test_get_size_returns_correct_disk_size_for_raid_4(self):
520+ node = factory.make_Node()
521+ small_size = random.randint(
522+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
523+ other_size = random.randint(small_size + 1, small_size + (10 ** 5))
524+ number_of_raid_devices = random.randint(2, 9)
525+ filesystems = [
526+ factory.make_Filesystem(
527+ fstype=FILESYSTEM_TYPE.RAID,
528+ block_device=factory.make_PhysicalBlockDevice(
529+ node=node, size=small_size)),
530+ ]
531+ for _ in range(number_of_raid_devices):
532+ filesystems.append(
533+ factory.make_Filesystem(
534+ fstype=FILESYSTEM_TYPE.RAID,
535+ block_device=factory.make_PhysicalBlockDevice(
536+ node=node, size=other_size)))
537+ # Spares are ignored and not taken into calculation.
538+ for _ in range(3):
539+ filesystems.append(
540+ factory.make_Filesystem(
541+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
542+ block_device=factory.make_PhysicalBlockDevice(
543+ node=node, size=other_size)))
544+ fsgroup = factory.make_FilesystemGroup(
545+ group_type=FILESYSTEM_GROUP_TYPE.RAID_4, filesystems=filesystems)
546+ self.assertEquals(
547+ small_size * number_of_raid_devices, fsgroup.get_size())
548+
549+ def test_get_size_returns_correct_disk_size_for_raid_5(self):
550+ node = factory.make_Node()
551+ small_size = random.randint(
552+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
553+ other_size = random.randint(small_size + 1, small_size + (10 ** 5))
554+ number_of_raid_devices = random.randint(2, 9)
555+ filesystems = [
556+ factory.make_Filesystem(
557+ fstype=FILESYSTEM_TYPE.RAID,
558+ block_device=factory.make_PhysicalBlockDevice(
559+ node=node, size=small_size)),
560+ ]
561+ for _ in range(number_of_raid_devices):
562+ filesystems.append(
563+ factory.make_Filesystem(
564+ fstype=FILESYSTEM_TYPE.RAID,
565+ block_device=factory.make_PhysicalBlockDevice(
566+ node=node, size=other_size)))
567+ # Spares are ignored and not taken into calculation.
568+ for _ in range(3):
569+ filesystems.append(
570+ factory.make_Filesystem(
571+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
572+ block_device=factory.make_PhysicalBlockDevice(
573+ node=node, size=other_size)))
574+ fsgroup = factory.make_FilesystemGroup(
575+ group_type=FILESYSTEM_GROUP_TYPE.RAID_5, filesystems=filesystems)
576+ self.assertEquals(
577+ small_size * number_of_raid_devices, fsgroup.get_size())
578+
579+ def test_get_size_returns_correct_disk_size_for_raid_6(self):
580+ node = factory.make_Node()
581+ small_size = random.randint(
582+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
583+ other_size = random.randint(small_size + 1, small_size + (10 ** 5))
584+ number_of_raid_devices = random.randint(3, 9)
585+ filesystems = [
586+ factory.make_Filesystem(
587+ fstype=FILESYSTEM_TYPE.RAID,
588+ block_device=factory.make_PhysicalBlockDevice(
589+ node=node, size=small_size)),
590+ ]
591+ for _ in range(number_of_raid_devices):
592+ filesystems.append(
593+ factory.make_Filesystem(
594+ fstype=FILESYSTEM_TYPE.RAID,
595+ block_device=factory.make_PhysicalBlockDevice(
596+ node=node, size=other_size)))
597+ # Spares are ignored and not taken into calculation.
598+ for _ in range(3):
599+ filesystems.append(
600+ factory.make_Filesystem(
601+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
602+ block_device=factory.make_PhysicalBlockDevice(
603+ node=node, size=other_size)))
604+ fsgroup = factory.make_FilesystemGroup(
605+ group_type=FILESYSTEM_GROUP_TYPE.RAID_6, filesystems=filesystems)
606+ self.assertEquals(
607+ small_size * (number_of_raid_devices - 1), fsgroup.get_size())
608+
609+ def test_get_size_returns_0_if_bcache_without_backing(self):
610+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.BCACHE)
611+ self.assertEquals(0, fsgroup.get_size())
612+
613+ def test_get_size_returns_size_of_backing_device_with_bcache(self):
614+ node = factory.make_Node()
615+ backing_size = random.randint(
616+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
617+ cache_size = random.randint(
618+ MIN_BLOCK_DEVICE_SIZE, MIN_BLOCK_DEVICE_SIZE ** 2)
619+ backing_block_device = factory.make_PhysicalBlockDevice(
620+ node=node, size=backing_size)
621+ cache_block_device = factory.make_PhysicalBlockDevice(
622+ node=node, size=cache_size)
623+ filesystems = [
624+ factory.make_Filesystem(
625+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
626+ block_device=cache_block_device),
627+ factory.make_Filesystem(
628+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
629+ block_device=backing_block_device),
630+ ]
631+ fsgroup = factory.make_FilesystemGroup(
632+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE, filesystems=filesystems)
633+ self.assertEquals(backing_size, fsgroup.get_size())
634+
635+ def test_is_lvm_returns_true_when_LVM_VG(self):
636+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.LVM_VG)
637+ self.assertTrue(fsgroup.is_lvm())
638+
639+ def test_is_lvm_returns_false_when_not_LVM_VG(self):
640+ fsgroup = FilesystemGroup(
641+ group_type=factory.pick_enum(
642+ FILESYSTEM_GROUP_TYPE, but_not=FILESYSTEM_GROUP_TYPE.LVM_VG))
643+ self.assertFalse(fsgroup.is_lvm())
644+
645+ def test_is_raid_returns_true_for_all_raid_types(self):
646 fsgroup = FilesystemGroup()
647- self.assertEquals(0, fsgroup.get_size())
648+ for raid_type in FILESYSTEM_GROUP_RAID_TYPES:
649+ fsgroup.group_type = raid_type
650+ self.assertTrue(
651+ fsgroup.is_raid(),
652+ "is_raid should return true for %s" % raid_type)
653+
654+ def test_is_raid_returns_false_for_LVM_VG(self):
655+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.LVM_VG)
656+ self.assertFalse(fsgroup.is_raid())
657+
658+ def test_is_raid_returns_false_for_BCACHE(self):
659+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.BCACHE)
660+ self.assertFalse(fsgroup.is_raid())
661+
662+ def test_is_bcache_returns_true_when_BCACHE(self):
663+ fsgroup = FilesystemGroup(group_type=FILESYSTEM_GROUP_TYPE.BCACHE)
664+ self.assertTrue(fsgroup.is_bcache())
665+
666+ def test_is_bcache_returns_false_when_not_BCACHE(self):
667+ fsgroup = FilesystemGroup(
668+ group_type=factory.pick_enum(
669+ FILESYSTEM_GROUP_TYPE, but_not=FILESYSTEM_GROUP_TYPE.BCACHE))
670+ self.assertFalse(fsgroup.is_bcache())
671
672 def test_can_save_new_filesystem_group_without_filesystems(self):
673 fsgroup = FilesystemGroup(
674@@ -86,18 +285,385 @@
675 fsgroup.save()
676
677 def test_cannot_save_without_filesystems_from_different_nodes(self):
678- fsgroup = FilesystemGroup(
679- group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
680- name=factory.make_name("vg"))
681- fsgroup.save()
682- fsgroup.filesystems.add(factory.make_Filesystem())
683- fsgroup.filesystems.add(factory.make_Filesystem())
684+ filesystems = [
685+ factory.make_Filesystem(),
686+ factory.make_Filesystem(),
687+ ]
688 with ExpectedException(
689 ValidationError,
690 re.escape(
691 "{'__all__': [u'All added filesystems must belong to "
692 "the same node.']}")):
693- fsgroup.save()
694+ factory.make_FilesystemGroup(
695+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
696+ filesystems=filesystems)
697+
698+ def test_cannot_save_volume_group_if_invalid_filesystem(self):
699+ node = factory.make_Node()
700+ filesystems = [
701+ factory.make_Filesystem(
702+ fstype=FILESYSTEM_TYPE.LVM_PV,
703+ block_device=factory.make_PhysicalBlockDevice(node=node)),
704+ factory.make_Filesystem(
705+ fstype=FILESYSTEM_TYPE.RAID,
706+ block_device=factory.make_PhysicalBlockDevice(node=node)),
707+ ]
708+ with ExpectedException(
709+ ValidationError,
710+ re.escape(
711+ "{'__all__': [u'Volume group can only contain lvm "
712+ "physical volumes.']}")):
713+ factory.make_FilesystemGroup(
714+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
715+ filesystems=filesystems)
716+
717+ def test_can_save_volume_group_if_valid_filesystems(self):
718+ node = factory.make_Node()
719+ filesystems = [
720+ factory.make_Filesystem(
721+ fstype=FILESYSTEM_TYPE.LVM_PV,
722+ block_device=factory.make_PhysicalBlockDevice(node=node)),
723+ factory.make_Filesystem(
724+ fstype=FILESYSTEM_TYPE.LVM_PV,
725+ block_device=factory.make_PhysicalBlockDevice(node=node)),
726+ ]
727+ # Test is that this does not raise an exception.
728+ factory.make_FilesystemGroup(
729+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
730+ filesystems=filesystems)
731+
732+ def test_cannot_save_raid_0_with_less_than_2_raid_devices(self):
733+ node = factory.make_Node()
734+ filesystems = [
735+ factory.make_Filesystem(
736+ fstype=FILESYSTEM_TYPE.RAID,
737+ block_device=factory.make_PhysicalBlockDevice(node=node)),
738+ ]
739+ with ExpectedException(
740+ ValidationError,
741+ re.escape(
742+ "{'__all__': [u'RAID level 0 must have exactly 2 raid "
743+ "devices and no spares.']}")):
744+ factory.make_FilesystemGroup(
745+ group_type=FILESYSTEM_GROUP_TYPE.RAID_0,
746+ filesystems=filesystems)
747+
748+ def test_cannot_save_raid_0_with_more_than_2_raid_devices(self):
749+ node = factory.make_Node()
750+ filesystems = [
751+ factory.make_Filesystem(
752+ fstype=FILESYSTEM_TYPE.RAID,
753+ block_device=factory.make_PhysicalBlockDevice(node=node))
754+ for _ in range(3)
755+ ]
756+ with ExpectedException(
757+ ValidationError,
758+ re.escape(
759+ "{'__all__': [u'RAID level 0 must have exactly 2 raid "
760+ "devices and no spares.']}")):
761+ factory.make_FilesystemGroup(
762+ group_type=FILESYSTEM_GROUP_TYPE.RAID_0,
763+ filesystems=filesystems)
764+
765+ def test_cannot_save_raid_0_with_spare_raid_devices(self):
766+ node = factory.make_Node()
767+ filesystems = [
768+ factory.make_Filesystem(
769+ fstype=FILESYSTEM_TYPE.RAID,
770+ block_device=factory.make_PhysicalBlockDevice(node=node))
771+ for _ in range(2)
772+ ]
773+ filesystems.append(
774+ factory.make_Filesystem(
775+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
776+ block_device=factory.make_PhysicalBlockDevice(node=node)))
777+ with ExpectedException(
778+ ValidationError,
779+ re.escape(
780+ "{'__all__': [u'RAID level 0 must have exactly 2 raid "
781+ "devices and no spares.']}")):
782+ factory.make_FilesystemGroup(
783+ group_type=FILESYSTEM_GROUP_TYPE.RAID_0,
784+ filesystems=filesystems)
785+
786+ def test_can_save_raid_0_with_exactly_2_raid_devices(self):
787+ node = factory.make_Node()
788+ filesystems = [
789+ factory.make_Filesystem(
790+ fstype=FILESYSTEM_TYPE.RAID,
791+ block_device=factory.make_PhysicalBlockDevice(node=node))
792+ for _ in range(2)
793+ ]
794+ # Test is that this does not raise an exception.
795+ factory.make_FilesystemGroup(
796+ group_type=FILESYSTEM_GROUP_TYPE.RAID_0,
797+ filesystems=filesystems)
798+
799+ def test_cannot_save_raid_1_with_less_than_2_raid_devices(self):
800+ node = factory.make_Node()
801+ filesystems = [
802+ factory.make_Filesystem(
803+ fstype=FILESYSTEM_TYPE.RAID,
804+ block_device=factory.make_PhysicalBlockDevice(node=node)),
805+ ]
806+ with ExpectedException(
807+ ValidationError,
808+ re.escape(
809+ "{'__all__': [u'RAID level 1 must have atleast 2 raid "
810+ "devices and no spares.']}")):
811+ factory.make_FilesystemGroup(
812+ group_type=FILESYSTEM_GROUP_TYPE.RAID_1,
813+ filesystems=filesystems)
814+
815+ def test_cannot_save_raid_1_with_spare_raid_devices(self):
816+ node = factory.make_Node()
817+ filesystems = [
818+ factory.make_Filesystem(
819+ fstype=FILESYSTEM_TYPE.RAID,
820+ block_device=factory.make_PhysicalBlockDevice(node=node))
821+ for _ in range(2)
822+ ]
823+ filesystems.append(
824+ factory.make_Filesystem(
825+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
826+ block_device=factory.make_PhysicalBlockDevice(node=node)))
827+ with ExpectedException(
828+ ValidationError,
829+ re.escape(
830+ "{'__all__': [u'RAID level 1 must have atleast 2 raid "
831+ "devices and no spares.']}")):
832+ factory.make_FilesystemGroup(
833+ group_type=FILESYSTEM_GROUP_TYPE.RAID_1,
834+ filesystems=filesystems)
835+
836+ def test_can_save_raid_1_with_2_or_more_raid_devices(self):
837+ node = factory.make_Node()
838+ filesystems = [
839+ factory.make_Filesystem(
840+ fstype=FILESYSTEM_TYPE.RAID,
841+ block_device=factory.make_PhysicalBlockDevice(node=node))
842+ for _ in range(random.randint(2, 10))
843+ ]
844+ # Test is that this does not raise an exception.
845+ factory.make_FilesystemGroup(
846+ group_type=FILESYSTEM_GROUP_TYPE.RAID_1,
847+ filesystems=filesystems)
848+
849+ def test_cannot_save_raid_4_with_less_than_3_raid_devices(self):
850+ node = factory.make_Node()
851+ filesystems = [
852+ factory.make_Filesystem(
853+ fstype=FILESYSTEM_TYPE.RAID,
854+ block_device=factory.make_PhysicalBlockDevice(node=node))
855+ for _ in range(random.randint(1, 2))
856+ ]
857+ with ExpectedException(
858+ ValidationError,
859+ re.escape(
860+ "{'__all__': [u'RAID level 4 must have atleast 3 raid "
861+ "devices and any number of spares.']}")):
862+ factory.make_FilesystemGroup(
863+ group_type=FILESYSTEM_GROUP_TYPE.RAID_4,
864+ filesystems=filesystems)
865+
866+ def test_can_save_raid_4_with_3_or_more_raid_devices_and_spares(self):
867+ node = factory.make_Node()
868+ filesystems = [
869+ factory.make_Filesystem(
870+ fstype=FILESYSTEM_TYPE.RAID,
871+ block_device=factory.make_PhysicalBlockDevice(node=node))
872+ for _ in range(random.randint(3, 10))
873+ ]
874+ for _ in range(random.randint(1, 5)):
875+ filesystems.append(
876+ factory.make_Filesystem(
877+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
878+ block_device=factory.make_PhysicalBlockDevice(node=node)))
879+ # Test is that this does not raise an exception.
880+ factory.make_FilesystemGroup(
881+ group_type=FILESYSTEM_GROUP_TYPE.RAID_4,
882+ filesystems=filesystems)
883+
884+ def test_cannot_save_raid_5_with_less_than_3_raid_devices(self):
885+ node = factory.make_Node()
886+ filesystems = [
887+ factory.make_Filesystem(
888+ fstype=FILESYSTEM_TYPE.RAID,
889+ block_device=factory.make_PhysicalBlockDevice(node=node))
890+ for _ in range(random.randint(1, 2))
891+ ]
892+ with ExpectedException(
893+ ValidationError,
894+ re.escape(
895+ "{'__all__': [u'RAID level 5 must have atleast 3 raid "
896+ "devices and any number of spares.']}")):
897+ factory.make_FilesystemGroup(
898+ group_type=FILESYSTEM_GROUP_TYPE.RAID_5,
899+ filesystems=filesystems)
900+
901+ def test_can_save_raid_5_with_3_or_more_raid_devices_and_spares(self):
902+ node = factory.make_Node()
903+ filesystems = [
904+ factory.make_Filesystem(
905+ fstype=FILESYSTEM_TYPE.RAID,
906+ block_device=factory.make_PhysicalBlockDevice(node=node))
907+ for _ in range(random.randint(3, 10))
908+ ]
909+ for _ in range(random.randint(1, 5)):
910+ filesystems.append(
911+ factory.make_Filesystem(
912+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
913+ block_device=factory.make_PhysicalBlockDevice(node=node)))
914+ # Test is that this does not raise an exception.
915+ factory.make_FilesystemGroup(
916+ group_type=FILESYSTEM_GROUP_TYPE.RAID_5,
917+ filesystems=filesystems)
918+
919+ def test_cannot_save_raid_6_with_less_than_4_raid_devices(self):
920+ node = factory.make_Node()
921+ filesystems = [
922+ factory.make_Filesystem(
923+ fstype=FILESYSTEM_TYPE.RAID,
924+ block_device=factory.make_PhysicalBlockDevice(node=node))
925+ for _ in range(random.randint(1, 3))
926+ ]
927+ with ExpectedException(
928+ ValidationError,
929+ re.escape(
930+ "{'__all__': [u'RAID level 6 must have atleast 4 raid "
931+ "devices and any number of spares.']}")):
932+ factory.make_FilesystemGroup(
933+ group_type=FILESYSTEM_GROUP_TYPE.RAID_6,
934+ filesystems=filesystems)
935+
936+ def test_can_save_raid_6_with_4_or_more_raid_devices_and_spares(self):
937+ node = factory.make_Node()
938+ filesystems = [
939+ factory.make_Filesystem(
940+ fstype=FILESYSTEM_TYPE.RAID,
941+ block_device=factory.make_PhysicalBlockDevice(node=node))
942+ for _ in range(random.randint(4, 10))
943+ ]
944+ for _ in range(random.randint(1, 5)):
945+ filesystems.append(
946+ factory.make_Filesystem(
947+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
948+ block_device=factory.make_PhysicalBlockDevice(node=node)))
949+ # Test is that this does not raise an exception.
950+ factory.make_FilesystemGroup(
951+ group_type=FILESYSTEM_GROUP_TYPE.RAID_6,
952+ filesystems=filesystems)
953+
954+ def test_cannot_save_bcache_without_cache(self):
955+ node = factory.make_Node()
956+ filesystems = [
957+ factory.make_Filesystem(
958+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
959+ block_device=factory.make_PhysicalBlockDevice(node=node)),
960+ ]
961+ with ExpectedException(
962+ ValidationError,
963+ re.escape(
964+ "{'__all__': [u'Bcache must contain one cache and one "
965+ "backing device.']}")):
966+ factory.make_FilesystemGroup(
967+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE,
968+ filesystems=filesystems)
969+
970+ def test_cannot_save_bcache_without_backing(self):
971+ node = factory.make_Node()
972+ filesystems = [
973+ factory.make_Filesystem(
974+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
975+ block_device=factory.make_PhysicalBlockDevice(node=node)),
976+ ]
977+ with ExpectedException(
978+ ValidationError,
979+ re.escape(
980+ "{'__all__': [u'Bcache must contain one cache and one "
981+ "backing device.']}")):
982+ factory.make_FilesystemGroup(
983+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE,
984+ filesystems=filesystems)
985+
986+ def test_can_save_bcache_with_cache_and_backing(self):
987+ node = factory.make_Node()
988+ filesystems = [
989+ factory.make_Filesystem(
990+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
991+ block_device=factory.make_PhysicalBlockDevice(node=node)),
992+ factory.make_Filesystem(
993+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
994+ block_device=factory.make_PhysicalBlockDevice(node=node)),
995+ ]
996+ # Test is that this does not raise an exception.
997+ factory.make_FilesystemGroup(
998+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE,
999+ filesystems=filesystems)
1000+
1001+ def test_cannot_save_bcache_with_multiple_caches(self):
1002+ node = factory.make_Node()
1003+ filesystems = [
1004+ factory.make_Filesystem(
1005+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
1006+ block_device=factory.make_PhysicalBlockDevice(node=node))
1007+ for _ in range(random.randint(2, 10))
1008+ ]
1009+ filesystems.append(
1010+ factory.make_Filesystem(
1011+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
1012+ block_device=factory.make_PhysicalBlockDevice(node=node)))
1013+ with ExpectedException(
1014+ ValidationError,
1015+ re.escape(
1016+ "{'__all__': [u'Bcache must contain one cache and one "
1017+ "backing device.']}")):
1018+ factory.make_FilesystemGroup(
1019+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE,
1020+ filesystems=filesystems)
1021+
1022+ def test_cannot_save_bcache_with_multiple_backings(self):
1023+ node = factory.make_Node()
1024+ filesystems = [
1025+ factory.make_Filesystem(
1026+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
1027+ block_device=factory.make_PhysicalBlockDevice(node=node))
1028+ for _ in range(random.randint(2, 10))
1029+ ]
1030+ filesystems.append(
1031+ factory.make_Filesystem(
1032+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
1033+ block_device=factory.make_PhysicalBlockDevice(node=node)))
1034+ with ExpectedException(
1035+ ValidationError,
1036+ re.escape(
1037+ "{'__all__': [u'Bcache must contain one cache and one "
1038+ "backing device.']}")):
1039+ factory.make_FilesystemGroup(
1040+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE,
1041+ filesystems=filesystems)
1042+
1043+ def test_cannot_save_bcache_with_multiple_caches_and_backings(self):
1044+ node = factory.make_Node()
1045+ filesystems = [
1046+ factory.make_Filesystem(
1047+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
1048+ block_device=factory.make_PhysicalBlockDevice(node=node))
1049+ for _ in range(random.randint(2, 10))
1050+ ]
1051+ for _ in range(random.randint(2, 10)):
1052+ filesystems.append(
1053+ factory.make_Filesystem(
1054+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
1055+ block_device=factory.make_PhysicalBlockDevice(node=node)))
1056+ with ExpectedException(
1057+ ValidationError,
1058+ re.escape(
1059+ "{'__all__': [u'Bcache must contain one cache and one "
1060+ "backing device.']}")):
1061+ factory.make_FilesystemGroup(
1062+ group_type=FILESYSTEM_GROUP_TYPE.BCACHE,
1063+ filesystems=filesystems)
1064
1065 def test_save_doesnt_overwrite_uuid(self):
1066 uuid = uuid4()
1067
1068=== modified file 'src/maasserver/models/tests/test_virtualblockdevice.py'
1069--- src/maasserver/models/tests/test_virtualblockdevice.py 2015-03-12 03:51:19 +0000
1070+++ src/maasserver/models/tests/test_virtualblockdevice.py 2015-07-01 21:29:38 +0000
1071@@ -18,8 +18,10 @@
1072 from uuid import uuid4
1073
1074 from django.core.exceptions import ValidationError
1075+from maasserver.enum import FILESYSTEM_GROUP_TYPE
1076 from maasserver.testing.factory import factory
1077 from maasserver.testing.testcase import MAASServerTestCase
1078+from maasserver.utils.converters import human_readable_bytes
1079 from testtools import ExpectedException
1080
1081
1082@@ -41,6 +43,26 @@
1083 "filesystem_group.']}")):
1084 block_device.save()
1085
1086+ def test_cannot_save_if_size_larger_than_volume_group(self):
1087+ filesystem_group = factory.make_FilesystemGroup(
1088+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG)
1089+ factory.make_VirtualBlockDevice(
1090+ filesystem_group=filesystem_group,
1091+ size=filesystem_group.get_size() / 2)
1092+ new_block_device_size = filesystem_group.get_size()
1093+ human_readable_size = human_readable_bytes(new_block_device_size)
1094+ with ExpectedException(
1095+ ValidationError,
1096+ re.escape(
1097+ "{'__all__': [u'There is not enough free space (%s) "
1098+ "on volume group %s.']}" % (
1099+ human_readable_size,
1100+ filesystem_group.name,
1101+ ))):
1102+ factory.make_VirtualBlockDevice(
1103+ filesystem_group=filesystem_group,
1104+ size=new_block_device_size)
1105+
1106 def test_save_doesnt_overwrite_uuid(self):
1107 uuid = uuid4()
1108 block_device = factory.make_VirtualBlockDevice(uuid=uuid)
1109
1110=== modified file 'src/maasserver/models/virtualblockdevice.py'
1111--- src/maasserver/models/virtualblockdevice.py 2015-06-30 20:51:24 +0000
1112+++ src/maasserver/models/virtualblockdevice.py 2015-07-01 21:29:38 +0000
1113@@ -73,12 +73,15 @@
1114 "Node must be the same node as the filesystem_group.")
1115
1116 # Check if the size of this is not larger than the free size of
1117- # its filesystem group.
1118- if self.size > self.filesystem_group.get_lvm_free_space():
1119+ # its filesystem group if its lvm.
1120+ if (self.filesystem_group.is_lvm() and
1121+ self.size > self.filesystem_group.get_lvm_free_space()):
1122 raise ValidationError(
1123 "There is not enough free space (%s) "
1124- "on filesystem_group %s." % (human_readable_bytes(self.size),
1125- self.filesystem_group.name))
1126+ "on volume group %s." % (
1127+ human_readable_bytes(self.size),
1128+ self.filesystem_group.name,
1129+ ))
1130
1131 def save(self, *args, **kwargs):
1132 if not self.uuid:
1133
1134=== modified file 'src/maasserver/testing/factory.py'
1135--- src/maasserver/testing/factory.py 2015-06-30 23:39:35 +0000
1136+++ src/maasserver/testing/factory.py 2015-07-01 21:29:38 +0000
1137@@ -29,6 +29,7 @@
1138 from maasserver.enum import (
1139 BOOT_RESOURCE_FILE_TYPE,
1140 BOOT_RESOURCE_TYPE,
1141+ FILESYSTEM_GROUP_RAID_TYPES,
1142 FILESYSTEM_GROUP_TYPE,
1143 FILESYSTEM_TYPE,
1144 INTERFACE_TYPE,
1145@@ -1236,14 +1237,16 @@
1146 def make_FilesystemGroup(
1147 self, uuid=None, group_type=None, name=None, create_params=None,
1148 filesystems=None, node=None, block_device_size=None,
1149- num_devices=4):
1150+ num_lvm_devices=4):
1151 if group_type is None:
1152 group_type = self.pick_enum(FILESYSTEM_GROUP_TYPE)
1153- if group_type == FILESYSTEM_GROUP_TYPE.LVM_VG:
1154- fstype = FILESYSTEM_TYPE.LVM_PV
1155 if name is None:
1156 if group_type == FILESYSTEM_GROUP_TYPE.LVM_VG:
1157 name = self.make_name("vg")
1158+ elif group_type in FILESYSTEM_GROUP_RAID_TYPES:
1159+ name = self.make_name("raid")
1160+ elif group_type == FILESYSTEM_GROUP_TYPE.BCACHE:
1161+ name = self.make_name("bcache")
1162 group = FilesystemGroup(
1163 uuid=uuid, group_type=group_type, name=name,
1164 create_params=create_params)
1165@@ -1251,12 +1254,64 @@
1166 if filesystems is None:
1167 if node is None:
1168 node = self.make_Node()
1169- for _ in range(num_devices):
1170- block_device = self.make_PhysicalBlockDevice(
1171- node, size=block_device_size)
1172- filesystem = self.make_Filesystem(
1173- fstype=fstype, block_device=block_device)
1174- group.filesystems.add(filesystem)
1175+ if group_type == FILESYSTEM_GROUP_TYPE.LVM_VG:
1176+ for _ in range(num_lvm_devices):
1177+ block_device = self.make_PhysicalBlockDevice(
1178+ node, size=block_device_size)
1179+ filesystem = self.make_Filesystem(
1180+ fstype=FILESYSTEM_TYPE.LVM_PV,
1181+ block_device=block_device)
1182+ group.filesystems.add(filesystem)
1183+ elif group_type == FILESYSTEM_GROUP_TYPE.RAID_0:
1184+ for _ in range(2):
1185+ block_device = self.make_PhysicalBlockDevice(node)
1186+ filesystem = self.make_Filesystem(
1187+ fstype=FILESYSTEM_TYPE.RAID,
1188+ block_device=block_device)
1189+ group.filesystems.add(filesystem)
1190+ elif group_type == FILESYSTEM_GROUP_TYPE.RAID_1:
1191+ for _ in range(2):
1192+ block_device = self.make_PhysicalBlockDevice(node)
1193+ filesystem = self.make_Filesystem(
1194+ fstype=FILESYSTEM_TYPE.RAID,
1195+ block_device=block_device)
1196+ group.filesystems.add(filesystem)
1197+ elif (group_type == FILESYSTEM_GROUP_TYPE.RAID_4 or
1198+ group_type == FILESYSTEM_GROUP_TYPE.RAID_5):
1199+ for _ in range(3):
1200+ block_device = self.make_PhysicalBlockDevice(node)
1201+ filesystem = self.make_Filesystem(
1202+ fstype=FILESYSTEM_TYPE.RAID,
1203+ block_device=block_device)
1204+ group.filesystems.add(filesystem)
1205+ spare_block_device = self.make_PhysicalBlockDevice(node)
1206+ spare_filesystem = self.make_Filesystem(
1207+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
1208+ block_device=spare_block_device)
1209+ group.filesystems.add(spare_filesystem)
1210+ elif group_type == FILESYSTEM_GROUP_TYPE.RAID_6:
1211+ for _ in range(4):
1212+ block_device = self.make_PhysicalBlockDevice(node)
1213+ filesystem = self.make_Filesystem(
1214+ fstype=FILESYSTEM_TYPE.RAID,
1215+ block_device=block_device)
1216+ group.filesystems.add(filesystem)
1217+ spare_block_device = self.make_PhysicalBlockDevice(node)
1218+ spare_filesystem = self.make_Filesystem(
1219+ fstype=FILESYSTEM_TYPE.RAID_SPARE,
1220+ block_device=spare_block_device)
1221+ group.filesystems.add(spare_filesystem)
1222+ elif group_type == FILESYSTEM_GROUP_TYPE.BCACHE:
1223+ cache_block_device = self.make_PhysicalBlockDevice(node)
1224+ cache_filesystem = self.make_Filesystem(
1225+ fstype=FILESYSTEM_TYPE.BCACHE_CACHE,
1226+ block_device=cache_block_device)
1227+ group.filesystems.add(cache_filesystem)
1228+ backing_block_device = self.make_PhysicalBlockDevice(node)
1229+ backing_filesystem = self.make_Filesystem(
1230+ fstype=FILESYSTEM_TYPE.BCACHE_BACKING,
1231+ block_device=backing_block_device)
1232+ group.filesystems.add(backing_filesystem)
1233 else:
1234 for filesystem in filesystems:
1235 group.filesystems.add(filesystem)
1236@@ -1278,10 +1333,19 @@
1237 tags = [self.make_name("tag") for _ in range(3)]
1238 if filesystem_group is None:
1239 filesystem_group = self.make_FilesystemGroup(
1240- node=node, block_device_size=size, num_devices=2)
1241+ node=node,
1242+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
1243+ block_device_size=size,
1244+ num_lvm_devices=2)
1245+ elif not filesystem_group.is_lvm():
1246+ raise RuntimeError(
1247+ "make_VirtualBlockDevice should only be used with "
1248+ "filesystem_group that has a group_type of LVM_VG. "
1249+ "If you need a VirtualBlockDevice that is for another type "
1250+ "use make_FilesystemGroup which will create a "
1251+ "VirtualBlockDevice automatically.")
1252 if name is None:
1253- if filesystem_group.group_type == FILESYSTEM_GROUP_TYPE.LVM_VG:
1254- name = self.make_name("lv")
1255+ name = self.make_name("device")
1256 if path is None:
1257 path = "/dev/mapper/%s-%s" % (filesystem_group.name, name)
1258
1259
1260=== modified file 'src/maasserver/tests/test_forms_blockdevice.py'
1261--- src/maasserver/tests/test_forms_blockdevice.py 2015-06-10 17:37:50 +0000
1262+++ src/maasserver/tests/test_forms_blockdevice.py 2015-07-01 21:29:38 +0000
1263@@ -16,7 +16,10 @@
1264
1265 import uuid
1266
1267-from maasserver.enum import FILESYSTEM_TYPE
1268+from maasserver.enum import (
1269+ FILESYSTEM_GROUP_TYPE,
1270+ FILESYSTEM_TYPE,
1271+)
1272 from maasserver.forms import (
1273 FormatBlockDeviceForm,
1274 MountBlockDeviceForm,
1275@@ -158,7 +161,9 @@
1276 block_device = factory.make_PhysicalBlockDevice()
1277 filesystem = factory.make_Filesystem(
1278 block_device=block_device, fstype=FILESYSTEM_TYPE.LVM_PV)
1279- factory.make_FilesystemGroup(filesystems=[filesystem])
1280+ factory.make_FilesystemGroup(
1281+ group_type=FILESYSTEM_GROUP_TYPE.LVM_VG,
1282+ filesystems=[filesystem])
1283 data = {
1284 'mount_point': factory.make_absolute_path(),
1285 }
1286
1287=== modified file 'src/maasserver/utils/version.py'
1288--- src/maasserver/utils/version.py 2015-04-07 13:27:55 +0000
1289+++ src/maasserver/utils/version.py 2015-07-01 21:29:38 +0000
1290@@ -78,7 +78,7 @@
1291 def simple_cache(fun):
1292 def wrapped(*args, **kwargs):
1293 key = hash(repr(fun) + repr(args) + repr(kwargs))
1294- if not key in _cache:
1295+ if key not in _cache:
1296 _cache[key] = fun(*args, **kwargs)
1297 return _cache[key]
1298
1299
1300=== modified file 'src/provisioningserver/tests/test_service_monitor.py'
1301--- src/provisioningserver/tests/test_service_monitor.py 2015-06-27 00:28:40 +0000
1302+++ src/provisioningserver/tests/test_service_monitor.py 2015-07-01 21:29:38 +0000
1303@@ -678,11 +678,13 @@
1304 with FakeLogger(
1305 "maas.service_monitor", level=logging.INFO) as maaslog:
1306 service_monitor._ensure_service(service)
1307- self.assertDocTestMatches(
1308- """\
1309+ lint_sucks = (
1310+ service.service_name,
1311+ service.service_name,
1312+ SERVICE_STATE.OFF,
1313+ "waiting",
1314+ )
1315+ self.assertDocTestMatches("""\
1316 Service '%s' is not on, it will be started.
1317 Service '%s' failed to start. Its current state is '%s' and '%s'.
1318- """ % (
1319- service.service_name, service.service_name,
1320- SERVICE_STATE.OFF, "waiting"),
1321- maaslog.output)
1322+ """ % lint_sucks, maaslog.output)