Merge lp:~trapnine/maas/rackd-updates-last-image-sync into lp:~maas-committers/maas/trunk
- rackd-updates-last-image-sync
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Jeffrey C Jones |
Approved revision: | no longer in the source branch. |
Merged at revision: | 5020 |
Proposed branch: | lp:~trapnine/maas/rackd-updates-last-image-sync |
Merge into: | lp:~maas-committers/maas/trunk |
Diff against target: |
574 lines (+193/-101) 10 files modified
src/maasserver/clusterrpc/boot_images.py (+0/-13) src/maasserver/clusterrpc/tests/test_boot_images.py (+0/-74) src/maasserver/rpc/rackcontrollers.py (+13/-1) src/maasserver/rpc/regionservice.py (+12/-0) src/maasserver/rpc/tests/test_rackcontrollers.py (+12/-0) src/provisioningserver/import_images/boot_resources.py (+6/-5) src/provisioningserver/import_images/tests/test_boot_resources.py (+6/-1) src/provisioningserver/rpc/boot_images.py (+44/-4) src/provisioningserver/rpc/region.py (+15/-0) src/provisioningserver/rpc/tests/test_boot_images.py (+85/-3) |
To merge this branch: | bzr merge lp:~trapnine/maas/rackd-updates-last-image-sync |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jeffrey C Jones (community) | Approve | ||
Gavin Panella (community) | Needs Fixing | ||
Review via email: mp+293988@code.launchpad.net |
Commit message
Racks will ask region to update last_image_sync directly.
Description of the change
Racks will ask region to update last_image_sync directly.
Lee Trager (ltrager) wrote : | # |
Jeffrey C Jones (trapnine) wrote : | # |
Thanks - everything addressed, please take a look.
Gavin Panella (allenap) wrote : | # |
Something I missed when talking to you yesterday is: this won't work :)
However, not a lot needs changing; see my comments for how I would go
about it.
In general, the problem here could be approached in two ways:
1. Record the image sync time in the database.
2. Ask racks for their last image sync time when needed.
#2 is preferable because then you're asking a primary source. The rack
may not be connected when you ask; however, when it last synced images
is not likely to be interesting information when it's down.
In #1's favour, MAAS relies on LISTEN/NOTIFY for disseminating change
within MAAS, so putting this value into the database makes sense.
However, #1, as in this proposal, can fail to update the sync time and
thus give a false reading. Fortunately racks do periodically import
images on their own, and so the last sync time will eventually converge
on truth.
Gavin Panella (allenap) wrote : | # |
Drat, I wrote a long comment in the diff that hasn't been saved; in fact it has been lost. I will write it again.
Gavin Panella (allenap) wrote : | # |
Ha, *now* it has saved one of my comments. Sadly it's by far the shorter of the two.
Gavin Panella (allenap) wrote : | # |
Long comment rewritten shorter.
Gavin Panella (allenap) wrote : | # |
This looks good, but I think you need to resurrect those end-to-end tests and get them passing. I'll help you with that. They were failing for a reason, and I suspect that reason was because things needed to be changed around.
Jeffrey C Jones (trapnine) wrote : | # |
Tests added and working.
MAAS Lander (maas-lander) wrote : | # |
The attempt to merge lp:~trapnine/maas/rackd-updates-last-image-sync into lp:maas failed. Below is the output from the failed tests.
Hit:1 http://
Get:2 http://
Hit:3 http://
Get:4 http://
Get:5 http://
Get:6 http://
Get:7 http://
Get:8 http://
Get:9 http://
Fetched 377 kB in 5s (72.9 kB/s)
Reading package lists...
sudo DEBIAN_
--no-
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubun
distro-info is already the newest version (0.14build1).
freeipmi-to...
Gavin Panella (allenap) wrote : | # |
You missed one comment from my last review:
> import_images() can return False in three situations, and True in one.
> Each should be tested. There should already be tests for those
> early-returns and run-to-completion so you'll just have to add an
> assert.
Do you mind fixing that in a follow-up?
Preview Diff
1 | === modified file 'src/maasserver/clusterrpc/boot_images.py' |
2 | --- src/maasserver/clusterrpc/boot_images.py 2016-04-14 19:54:36 +0000 |
3 | +++ src/maasserver/clusterrpc/boot_images.py 2016-05-12 17:26:13 +0000 |
4 | @@ -23,7 +23,6 @@ |
5 | BootResource, |
6 | RackController, |
7 | ) |
8 | -from maasserver.models.timestampedmodel import now |
9 | from maasserver.rpc import ( |
10 | getAllClients, |
11 | getClientFor, |
12 | @@ -289,11 +288,6 @@ |
13 | for system_id in self.system_ids), |
14 | consumeErrors=True) |
15 | |
16 | - @staticmethod |
17 | - def touch_last_image_sync(system_ids): |
18 | - RackController.objects.filter( |
19 | - system_id__in=system_ids).update(last_image_sync=now()) |
20 | - |
21 | @asynchronous |
22 | def run(self, concurrency=1): |
23 | """Ask the rack controllers to download the region's boot resources. |
24 | @@ -314,20 +308,13 @@ |
25 | "Rack controller (%s) did not import boot resources; it is " |
26 | "not connected to the region at this time." |
27 | ) |
28 | - successes = [] |
29 | for system_id, (success, result) in zip(self.system_ids, results): |
30 | if success: |
31 | - successes.append(system_id) |
32 | log.msg(message_success % system_id) |
33 | elif result.check(NoConnectionsAvailable): |
34 | log.msg(message_disconn % system_id) |
35 | else: |
36 | log.err(result, message_failure % system_id) |
37 | |
38 | - return deferToDatabase( |
39 | - RackControllersImporter.touch_last_image_sync, |
40 | - successes).addErrback( |
41 | - log.err, "Failed to touch last image sync timestamps.") |
42 | - |
43 | return self(lock).addCallback(report).addErrback( |
44 | log.err, "General failure syncing boot resources.") |
45 | |
46 | === modified file 'src/maasserver/clusterrpc/tests/test_boot_images.py' |
47 | --- src/maasserver/clusterrpc/tests/test_boot_images.py 2016-05-06 11:41:05 +0000 |
48 | +++ src/maasserver/clusterrpc/tests/test_boot_images.py 2016-05-12 17:26:13 +0000 |
49 | @@ -5,7 +5,6 @@ |
50 | |
51 | __all__ = [] |
52 | |
53 | -from datetime import timedelta |
54 | import os |
55 | import random |
56 | from unittest.mock import ( |
57 | @@ -26,7 +25,6 @@ |
58 | from maasserver.clusterrpc.testing.boot_images import make_rpc_boot_image |
59 | from maasserver.enum import BOOT_RESOURCE_TYPE |
60 | from maasserver.models.config import Config |
61 | -from maasserver.models.timestampedmodel import now |
62 | from maasserver.rpc import getAllClients |
63 | from maasserver.rpc.testing.fixtures import ( |
64 | MockLiveRegionToClusterRPCFixture, |
65 | @@ -38,7 +36,6 @@ |
66 | ) |
67 | from maasserver.testing.factory import factory |
68 | from maasserver.testing.testcase import MAASServerTestCase |
69 | -from maasserver.utils.orm import reload_object |
70 | from maastesting.matchers import ( |
71 | MockCalledOnceWith, |
72 | MockCallsMatch, |
73 | @@ -597,74 +594,3 @@ |
74 | connected to the region at this time. |
75 | """ % (rack_1.system_id, rack_2.system_id, rack_3.system_id), |
76 | logger.output) |
77 | - |
78 | - def test__run_touches_synced_racks_last_image_sync(self): |
79 | - # Avoid deferring to the database. |
80 | - self.patch(boot_images_module, "deferToDatabase", maybeDeferred) |
81 | - touch = self.patch_autospec( |
82 | - RackControllersImporter, 'touch_last_image_sync') |
83 | - |
84 | - # Some clusters that we'll ask to import resources. |
85 | - rack_1 = factory.make_RackController() |
86 | - rack_2 = factory.make_RackController() |
87 | - rack_3 = factory.make_RackController() |
88 | - # Cluster #1 will work fine. |
89 | - cluster_1 = self.rpc.makeCluster(rack_1, ImportBootImages) |
90 | - cluster_1.ImportBootImages.return_value = succeed({}) |
91 | - # Cluster #2 will break. |
92 | - cluster_2 = self.rpc.makeCluster(rack_2, ImportBootImages) |
93 | - cluster_2.ImportBootImages.return_value = fail(ZeroDivisionError()) |
94 | - # Cluster #3 is not connected. |
95 | - |
96 | - # Do the import. |
97 | - importer = RackControllersImporter.new([ |
98 | - rack_1.system_id, |
99 | - rack_2.system_id, |
100 | - rack_3.system_id, |
101 | - ]) |
102 | - importer.run().wait(5) |
103 | - |
104 | - # Make sure that only rack_1's last_sync_time would get touched |
105 | - self.assertThat(touch, MockCalledOnceWith([rack_1.system_id])) |
106 | - |
107 | - def test__broken_touch_last_image_sync_reports_results(self): |
108 | - # Avoid deferring to the database. |
109 | - self.patch(boot_images_module, "deferToDatabase", maybeDeferred) |
110 | - # Make touch_last_image_sync fail. |
111 | - self.patch_autospec( |
112 | - RackControllersImporter, |
113 | - 'touch_last_image_sync').side_effect = Exception() |
114 | - |
115 | - rack_1 = factory.make_RackController() |
116 | - cluster_1 = self.rpc.makeCluster(rack_1, ImportBootImages) |
117 | - cluster_1.ImportBootImages.return_value = succeed({}) |
118 | - |
119 | - # Do the import with reporting. |
120 | - importer = RackControllersImporter.new([rack_1.system_id]) |
121 | - |
122 | - with TwistedLoggerFixture() as logger: |
123 | - importer.run().wait(5) |
124 | - |
125 | - self.assertDocTestMatches( |
126 | - """\ |
127 | - ... |
128 | - --- |
129 | - Failed to touch last image sync timestamps. |
130 | - ... |
131 | - """, logger.output) |
132 | - |
133 | - def test__touch_last_image_sync(self): |
134 | - # Create a few rackd's, two with no sync times. |
135 | - birthday = now() - timedelta(minutes=random.randint(1, 15)) |
136 | - rack_1 = factory.make_RackController(last_image_sync=None) |
137 | - rack_2 = factory.make_RackController(last_image_sync=None) |
138 | - rack_3 = factory.make_RackController(last_image_sync=birthday) |
139 | - rack_4 = factory.make_RackController(last_image_sync=birthday) |
140 | - |
141 | - # Touch cluster with and without sync_time, confirm properly updated. |
142 | - RackControllersImporter.touch_last_image_sync( |
143 | - [rack_1.system_id, rack_3.system_id]) |
144 | - self.assertIsNotNone(reload_object(rack_1).last_image_sync) |
145 | - self.assertIsNone(reload_object(rack_2).last_image_sync) |
146 | - self.assertGreater(reload_object(rack_3).last_image_sync, birthday) |
147 | - self.assertEqual(reload_object(rack_4).last_image_sync, birthday) |
148 | |
149 | === modified file 'src/maasserver/rpc/rackcontrollers.py' |
150 | --- src/maasserver/rpc/rackcontrollers.py 2016-05-06 07:49:56 +0000 |
151 | +++ src/maasserver/rpc/rackcontrollers.py 2016-05-12 17:26:13 +0000 |
152 | @@ -5,9 +5,9 @@ |
153 | |
154 | __all__ = [ |
155 | "register_rackcontroller", |
156 | + "update_last_image_sync", |
157 | ] |
158 | |
159 | - |
160 | from django.db import ( |
161 | IntegrityError, |
162 | transaction, |
163 | @@ -22,6 +22,7 @@ |
164 | StaticIPAddress, |
165 | ) |
166 | from maasserver.models.node import typecast_node |
167 | +from maasserver.models.timestampedmodel import now |
168 | from maasserver.utils.orm import transactional |
169 | from provisioningserver.logger import get_maas_logger |
170 | from provisioningserver.utils.twisted import synchronous |
171 | @@ -198,3 +199,14 @@ |
172 | """Update the interface definition on the rack controller.""" |
173 | rack_controller = RackController.objects.get(system_id=system_id) |
174 | rack_controller.update_interfaces(interfaces) |
175 | + |
176 | + |
177 | +@synchronous |
178 | +@transactional |
179 | +def update_last_image_sync(system_id): |
180 | + """Update rack controller's last_image_sync. |
181 | + |
182 | + for :py:class:`~provisioningserver.rpc.region.UpdateLastImageSync. |
183 | + """ |
184 | + RackController.objects.filter( |
185 | + system_id=system_id).update(last_image_sync=now()) |
186 | |
187 | === modified file 'src/maasserver/rpc/regionservice.py' |
188 | --- src/maasserver/rpc/regionservice.py 2016-04-29 22:46:46 +0000 |
189 | +++ src/maasserver/rpc/regionservice.py 2016-05-12 17:26:13 +0000 |
190 | @@ -291,6 +291,18 @@ |
191 | d.addCallback(lambda nodes: {"nodes": nodes}) |
192 | return d |
193 | |
194 | + @region.UpdateLastImageSync.responder |
195 | + def update_last_image_sync(self, system_id): |
196 | + """update_last_image_sync() |
197 | + |
198 | + Implementation of |
199 | + :py:class:`~provisioningserver.rpc.region.UpdateLastImageSync`. |
200 | + """ |
201 | + d = deferToDatabase( |
202 | + rackcontrollers.update_last_image_sync, system_id) |
203 | + d.addCallback(lambda args: {}) |
204 | + return d |
205 | + |
206 | @region.UpdateNodePowerState.responder |
207 | def update_node_power_state(self, system_id, power_state): |
208 | """update_node_power_state() |
209 | |
210 | === modified file 'src/maasserver/rpc/tests/test_rackcontrollers.py' |
211 | --- src/maasserver/rpc/tests/test_rackcontrollers.py 2016-05-09 23:54:19 +0000 |
212 | +++ src/maasserver/rpc/tests/test_rackcontrollers.py 2016-05-12 17:26:13 +0000 |
213 | @@ -21,12 +21,14 @@ |
214 | NodeGroupToRackController, |
215 | RackController, |
216 | ) |
217 | +from maasserver.models.timestampedmodel import now |
218 | from maasserver.rpc.rackcontrollers import ( |
219 | handle_upgrade, |
220 | register_new_rackcontroller, |
221 | register_rackcontroller, |
222 | update_foreign_dhcp, |
223 | update_interfaces, |
224 | + update_last_image_sync, |
225 | ) |
226 | from maasserver.testing.factory import factory |
227 | from maasserver.testing.testcase import MAASServerTestCase |
228 | @@ -322,3 +324,13 @@ |
229 | self.assertThat( |
230 | patched_update_interfaces, |
231 | MockCalledOnceWith(sentinel.interfaces)) |
232 | + |
233 | + |
234 | +class TestUpdateLastImageSync(MAASServerTestCase): |
235 | + |
236 | + def test__updates_last_image_sync(self): |
237 | + rack = factory.make_RackController() |
238 | + |
239 | + update_last_image_sync(rack.system_id) |
240 | + |
241 | + self.assertEqual(now(), reload_object(rack).last_image_sync) |
242 | |
243 | === modified file 'src/provisioningserver/import_images/boot_resources.py' |
244 | --- src/provisioningserver/import_images/boot_resources.py 2016-04-27 18:25:56 +0000 |
245 | +++ src/provisioningserver/import_images/boot_resources.py 2016-05-12 17:26:13 +0000 |
246 | @@ -32,6 +32,7 @@ |
247 | from provisioningserver.import_images.helpers import maaslog |
248 | from provisioningserver.import_images.keyrings import write_all_keyrings |
249 | from provisioningserver.import_images.product_mapping import map_products |
250 | +from provisioningserver.rpc import getRegionClient |
251 | from provisioningserver.service_monitor import service_monitor |
252 | from provisioningserver.utils.fs import ( |
253 | atomic_symlink, |
254 | @@ -40,6 +41,7 @@ |
255 | tempdir, |
256 | ) |
257 | from provisioningserver.utils.shell import call_and_check |
258 | +from twisted.internet.defer import inlineCallbacks |
259 | from twisted.python.filepath import FilePath |
260 | |
261 | |
262 | @@ -236,7 +238,7 @@ |
263 | maaslog.info("Started importing boot images.") |
264 | if len(sources) == 0: |
265 | maaslog.warning("Can't import: region did not provide a source.") |
266 | - return |
267 | + return False |
268 | |
269 | with tempdir('keyrings') as keyrings_path: |
270 | # XXX: Band-aid to ensure that the keyring_data is bytes. Future task: |
271 | @@ -256,7 +258,7 @@ |
272 | maaslog.warning( |
273 | "Finished importing boot images, the region does not have " |
274 | "any boot images available.") |
275 | - return |
276 | + return False |
277 | |
278 | with ClusterConfiguration.open() as config: |
279 | storage = FilePath(config.tftp_root).parent().path |
280 | @@ -265,7 +267,7 @@ |
281 | maaslog.info( |
282 | "Finished importing boot images, the region does not " |
283 | "have any new images.") |
284 | - return |
285 | + return False |
286 | |
287 | product_mapping = map_products(image_descriptions) |
288 | |
289 | @@ -290,6 +292,7 @@ |
290 | |
291 | # Import is now finished. |
292 | maaslog.info("Finished importing boot images.") |
293 | + return True |
294 | |
295 | |
296 | def main(args): |
297 | @@ -318,12 +321,10 @@ |
298 | import traceback |
299 | |
300 | from provisioningserver import services |
301 | - from provisioningserver.rpc import getRegionClient |
302 | from provisioningserver.rpc.clusterservice import ClusterClientService |
303 | from provisioningserver.rpc.exceptions import NoConnectionsAvailable |
304 | from provisioningserver.utils.twisted import retries, pause |
305 | from twisted.internet import reactor |
306 | - from twisted.internet.defer import inlineCallbacks |
307 | from twisted.internet.threads import deferToThread |
308 | |
309 | @inlineCallbacks |
310 | |
311 | === modified file 'src/provisioningserver/import_images/tests/test_boot_resources.py' |
312 | --- src/provisioningserver/import_images/tests/test_boot_resources.py 2016-05-06 11:41:05 +0000 |
313 | +++ src/provisioningserver/import_images/tests/test_boot_resources.py 2016-05-12 17:26:13 +0000 |
314 | @@ -28,7 +28,10 @@ |
315 | MockCalledWith, |
316 | MockCallsMatch, |
317 | ) |
318 | -from maastesting.testcase import MAASTestCase |
319 | +from maastesting.testcase import ( |
320 | + MAASTestCase, |
321 | + MAASTwistedRunTest, |
322 | +) |
323 | from maastesting.utils import age_file |
324 | from provisioningserver.boot import BootMethodRegistry |
325 | from provisioningserver.config import ( |
326 | @@ -563,6 +566,8 @@ |
327 | class TestImportImages(MAASTestCase): |
328 | """Tests for the `import_images`() function.""" |
329 | |
330 | + run_tests_with = MAASTwistedRunTest.make_factory(timeout=5) |
331 | + |
332 | def test_writes_source_keyrings(self): |
333 | # Stop import_images() from actually doing anything. |
334 | self.patch(boot_resources, 'maaslog') |
335 | |
336 | === modified file 'src/provisioningserver/rpc/boot_images.py' |
337 | --- src/provisioningserver/rpc/boot_images.py 2016-04-14 19:54:36 +0000 |
338 | +++ src/provisioningserver/rpc/boot_images.py 2016-05-12 17:26:13 +0000 |
339 | @@ -16,9 +16,19 @@ |
340 | from provisioningserver.boot import tftppath |
341 | from provisioningserver.config import ClusterConfiguration |
342 | from provisioningserver.import_images import boot_resources |
343 | -from provisioningserver.utils.env import environment_variables |
344 | +from provisioningserver.rpc import getRegionClient |
345 | +from provisioningserver.rpc.region import UpdateLastImageSync |
346 | +from provisioningserver.utils.env import ( |
347 | + environment_variables, |
348 | + get_maas_id, |
349 | +) |
350 | from provisioningserver.utils.twisted import synchronous |
351 | +from twisted.internet.defer import ( |
352 | + fail, |
353 | + inlineCallbacks, |
354 | +) |
355 | from twisted.internet.threads import deferToThread |
356 | +from twisted.python import log |
357 | |
358 | |
359 | CACHED_BOOT_IMAGES = None |
360 | @@ -103,22 +113,52 @@ |
361 | no_proxy_hosts += list(get_hosts_from_sources(sources)) |
362 | variables['no_proxy'] = ','.join(no_proxy_hosts) |
363 | with environment_variables(variables): |
364 | - boot_resources.import_images(sources) |
365 | + imported = boot_resources.import_images(sources) |
366 | |
367 | # Update the boot images cache so `list_boot_images` returns the |
368 | # correct information. |
369 | reload_boot_images() |
370 | |
371 | + # Tell callers if anything happened. |
372 | + return imported |
373 | + |
374 | |
375 | def import_boot_images(sources, http_proxy=None, https_proxy=None): |
376 | """Imports the boot images from the given sources.""" |
377 | lock = concurrency.boot_images |
378 | if not lock.locked: |
379 | return lock.run( |
380 | - deferToThread, _run_import, sources, |
381 | - http_proxy=http_proxy, https_proxy=https_proxy) |
382 | + _import_boot_images, sources, http_proxy=http_proxy, |
383 | + https_proxy=https_proxy) |
384 | + |
385 | + |
386 | +@inlineCallbacks |
387 | +def _import_boot_images(sources, http_proxy=None, https_proxy=None): |
388 | + """Import boot images then inform the region. |
389 | + |
390 | + Helper for `import_boot_images`. |
391 | + """ |
392 | + proxies = dict(http_proxy=http_proxy, https_proxy=https_proxy) |
393 | + imported = yield deferToThread(_run_import, sources, **proxies) |
394 | + if imported: |
395 | + yield touch_last_image_sync_timestamp().addErrback( |
396 | + log.err, "Failure touching last image sync timestamp.") |
397 | |
398 | |
399 | def is_import_boot_images_running(): |
400 | """Return True if the import process is currently running.""" |
401 | return concurrency.boot_images.locked |
402 | + |
403 | + |
404 | +def touch_last_image_sync_timestamp(): |
405 | + """Inform the region that images have just been synchronised. |
406 | + |
407 | + :return: :class:`Deferred` that can fail with `NoConnectionsAvailable` or |
408 | + any exception arising from an `UpdateLastImageSync` RPC. |
409 | + """ |
410 | + try: |
411 | + client = getRegionClient() |
412 | + except: |
413 | + return fail() |
414 | + else: |
415 | + return client(UpdateLastImageSync, system_id=get_maas_id()) |
416 | |
417 | === modified file 'src/provisioningserver/rpc/region.py' |
418 | --- src/provisioningserver/rpc/region.py 2016-04-11 16:23:26 +0000 |
419 | +++ src/provisioningserver/rpc/region.py 2016-05-12 17:26:13 +0000 |
420 | @@ -26,6 +26,7 @@ |
421 | "SendEvent", |
422 | "SendEventMACAddress", |
423 | "UpdateInterfaces", |
424 | + "UpdateLastImageSync", |
425 | "UpdateNodePowerState", |
426 | ] |
427 | |
428 | @@ -262,6 +263,20 @@ |
429 | } |
430 | |
431 | |
432 | +class UpdateLastImageSync(amp.Command): |
433 | + """Update Rack Controller's Last Image Sync. |
434 | + |
435 | + :since: 2.0 |
436 | + """ |
437 | + |
438 | + arguments = [ |
439 | + # A rack controller's system_id. |
440 | + (b'system_id', amp.Unicode()), |
441 | + ] |
442 | + response = [] |
443 | + errors = [] |
444 | + |
445 | + |
446 | class UpdateNodePowerState(amp.Command): |
447 | """Update Node Power State. |
448 | |
449 | |
450 | === modified file 'src/provisioningserver/rpc/tests/test_boot_images.py' |
451 | --- src/provisioningserver/rpc/tests/test_boot_images.py 2016-05-06 11:41:05 +0000 |
452 | +++ src/provisioningserver/rpc/tests/test_boot_images.py 2016-05-12 17:26:13 +0000 |
453 | @@ -21,7 +21,10 @@ |
454 | from provisioningserver import concurrency |
455 | from provisioningserver.boot import tftppath |
456 | from provisioningserver.import_images import boot_resources |
457 | -from provisioningserver.rpc import boot_images |
458 | +from provisioningserver.rpc import ( |
459 | + boot_images, |
460 | + region, |
461 | +) |
462 | from provisioningserver.rpc.boot_images import ( |
463 | _run_import, |
464 | fix_sources_for_cluster, |
465 | @@ -31,14 +34,23 @@ |
466 | list_boot_images, |
467 | reload_boot_images, |
468 | ) |
469 | +from provisioningserver.rpc.region import UpdateLastImageSync |
470 | +from provisioningserver.rpc.testing import MockLiveClusterToRegionRPCFixture |
471 | from provisioningserver.testing.config import ( |
472 | BootSourcesFixture, |
473 | ClusterConfigurationFixture, |
474 | ) |
475 | from provisioningserver.testing.testcase import PservTestCase |
476 | from provisioningserver.utils.twisted import pause |
477 | -from testtools.matchers import Equals |
478 | +from testtools.matchers import ( |
479 | + Equals, |
480 | + Is, |
481 | +) |
482 | from twisted.internet import defer |
483 | +from twisted.internet.defer import ( |
484 | + inlineCallbacks, |
485 | + succeed, |
486 | +) |
487 | from twisted.internet.task import Clock |
488 | |
489 | |
490 | @@ -205,7 +217,7 @@ |
491 | self.patch(boot_resources, 'logger') |
492 | self.patch(boot_resources, 'locate_config').return_value = ( |
493 | fixture.filename) |
494 | - self.assertIsNone(_run_import(sources=[])) |
495 | + self.assertThat(_run_import(sources=[]), Is(False)) |
496 | |
497 | def test__run_import_sets_GPGHOME(self): |
498 | home = factory.make_name('home') |
499 | @@ -288,6 +300,76 @@ |
500 | clock.advance(1) |
501 | self.assertFalse(concurrency.boot_images.locked) |
502 | |
503 | + @inlineCallbacks |
504 | + def test_update_last_image_sync(self): |
505 | + get_maas_id = self.patch(boot_images, "get_maas_id") |
506 | + get_maas_id.return_value = factory.make_string() |
507 | + getRegionClient = self.patch(boot_images, "getRegionClient") |
508 | + _run_import = self.patch_autospec(boot_images, '_run_import') |
509 | + _run_import.return_value = True |
510 | + yield boot_images._import_boot_images(sentinel.sources) |
511 | + self.assertThat( |
512 | + _run_import, MockCalledOnceWith(sentinel.sources, None, None)) |
513 | + self.assertThat(getRegionClient, MockCalledOnceWith()) |
514 | + self.assertThat(get_maas_id, MockCalledOnceWith()) |
515 | + client = getRegionClient.return_value |
516 | + self.assertThat( |
517 | + client, MockCalledOnceWith( |
518 | + UpdateLastImageSync, system_id=get_maas_id())) |
519 | + |
520 | + @inlineCallbacks |
521 | + def test_update_last_image_sync_not_performed(self): |
522 | + get_maas_id = self.patch(boot_images, "get_maas_id") |
523 | + get_maas_id.return_value = factory.make_string() |
524 | + getRegionClient = self.patch(boot_images, "getRegionClient") |
525 | + _run_import = self.patch_autospec(boot_images, '_run_import') |
526 | + _run_import.return_value = False |
527 | + yield boot_images._import_boot_images(sentinel.sources) |
528 | + self.assertThat( |
529 | + _run_import, MockCalledOnceWith(sentinel.sources, None, None)) |
530 | + self.assertThat(getRegionClient, MockNotCalled()) |
531 | + self.assertThat(get_maas_id, MockNotCalled()) |
532 | + |
533 | + @inlineCallbacks |
534 | + def test_update_last_image_sync_end_to_end(self): |
535 | + get_maas_id = self.patch(boot_images, "get_maas_id") |
536 | + get_maas_id.return_value = factory.make_string() |
537 | + self.useFixture(ClusterConfigurationFixture()) |
538 | + fixture = self.useFixture(MockLiveClusterToRegionRPCFixture()) |
539 | + protocol, connecting = fixture.makeEventLoop( |
540 | + region.UpdateLastImageSync) |
541 | + protocol.UpdateLastImageSync.return_value = succeed({}) |
542 | + self.addCleanup((yield connecting)) |
543 | + self.patch_autospec(boot_resources, 'import_images') |
544 | + boot_resources.import_images.return_value = True |
545 | + sources, hosts = make_sources() |
546 | + yield boot_images.import_boot_images(sources) |
547 | + self.assertThat( |
548 | + boot_resources.import_images, |
549 | + MockCalledOnceWith(sources)) |
550 | + self.assertThat( |
551 | + protocol.UpdateLastImageSync, |
552 | + MockCalledOnceWith(protocol, system_id=get_maas_id())) |
553 | + |
554 | + @inlineCallbacks |
555 | + def test_update_last_image_sync_end_to_end_import_not_performed(self): |
556 | + self.useFixture(ClusterConfigurationFixture()) |
557 | + fixture = self.useFixture(MockLiveClusterToRegionRPCFixture()) |
558 | + protocol, connecting = fixture.makeEventLoop( |
559 | + region.UpdateLastImageSync) |
560 | + protocol.UpdateLastImageSync.return_value = succeed({}) |
561 | + self.addCleanup((yield connecting)) |
562 | + self.patch_autospec(boot_resources, 'import_images') |
563 | + boot_resources.import_images.return_value = False |
564 | + sources, hosts = make_sources() |
565 | + yield boot_images.import_boot_images(sources) |
566 | + self.assertThat( |
567 | + boot_resources.import_images, |
568 | + MockCalledOnceWith(sources)) |
569 | + self.assertThat( |
570 | + protocol.UpdateLastImageSync, |
571 | + MockNotCalled()) |
572 | + |
573 | |
574 | class TestIsImportBootImagesRunning(PservTestCase): |
575 |
Looks good so far just a couple of comments below. As you previously stated there should be a way to get the connection ID, which is the system_id on the server side but I haven't been able to figure out how to do that.