Merge lp:~raharper/curtin/trunk.fix-raid-over-iscsi into lp:~curtin-dev/curtin/trunk

Proposed by Ryan Harper
Status: Merged
Merged at revision: 526
Proposed branch: lp:~raharper/curtin/trunk.fix-raid-over-iscsi
Merge into: lp:~curtin-dev/curtin/trunk
Diff against target: 154 lines (+95/-5)
5 files modified
curtin/block/iscsi.py (+9/-0)
curtin/commands/install.py (+17/-5)
curtin/util.py (+14/-0)
tests/unittests/test_util.py (+51/-0)
tests/vmtests/test_mdadm_iscsi.py (+4/-0)
To merge this branch: bzr merge lp:~raharper/curtin/trunk.fix-raid-over-iscsi
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Scott Moser (community) Approve
Review via email: mp+330309@code.launchpad.net

Description of the change

install: ensure iscsi service is running to handle shutdown properly

In some cases it is problematic to shutdown the iscsi targets (like raid
over iscsi). Instead of introducing extra work to stop multi-layer
storage devices during the install of curtin, this patch addresses the
core issue around iscsi termination; ensuring the iscsi service is active
prior to shutting down. In Artful, the open-iscsi service does not
automatically start if iscsi configuration is not present during boot;
curtin will restart/start the service if the storage configuration
contains iscsi disks.

This approach successfully passes all of the iscsi configurations
(plain, lvm over iscsi, and raid over iscsi).

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

./tools/jenkins-runner -p 4 tests/vmtests/test_mdadm_bcache.py tests/vmtests/test_mdadm_iscsi.py tests/vmtests/test_raid5_bcache.py tests/vmtests/test_iscsi.py tests/vmtests/test_lvm_iscsi.py

...

Ran 489 tests in 2308.432s

OK (SKIP=2)

Revision history for this message
Scott Moser (smoser) wrote :

so somewhere during block-meta iscsi will get started if not already running?

Revision history for this message
Ryan Harper (raharper) :
526. By Ryan Harper

merge from trunk

527. By Ryan Harper

uses_systemd: add caching, drop target_path use, ephemeral only

Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

mainly just need to somehow document this in code for the point later when someone says "what is going on here... we start a service just to handle shutdown ! ?"

528. By Ryan Harper

Add additional documentation in install.py on why we restart iscsi service

Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

this is fine.
my only comment is that 'sdpath' is wierd as an argument and i'd never expect anythign to use it.

i realize it is just for testing. Woudl it be better to just use 'os.path.isdir()' rather than the lstat? and then you could easily mock the response to that.

529. By Ryan Harper

Replace os.lstat with os.path.isdir; drop sdpath arg; adjust unittests

Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

i approve, just fix the vestigial 'sdpath' uses.

review: Approve
530. By Ryan Harper

Drop useless tmp_dir() in uses_systemd unittest

Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'curtin/block/iscsi.py'
--- curtin/block/iscsi.py 2017-08-31 19:22:08 +0000
+++ curtin/block/iscsi.py 2017-09-12 21:16:54 +0000
@@ -195,6 +195,15 @@
195 return target_nodes_location195 return target_nodes_location
196196
197197
198def restart_iscsi_service():
199 LOG.info('restarting iscsi service')
200 if util.uses_systemd():
201 cmd = ['systemctl', 'reload-or-restart', 'open-iscsi']
202 else:
203 cmd = ['service', 'open-iscsi', 'restart']
204 util.subp(cmd, capture=True)
205
206
198def save_iscsi_config(iscsi_disk):207def save_iscsi_config(iscsi_disk):
199 state = util.load_command_environment()208 state = util.load_command_environment()
200 # A nodes directory will be created in the same directory as the209 # A nodes directory will be created in the same directory as the
201210
=== modified file 'curtin/commands/install.py'
--- curtin/commands/install.py 2017-08-31 18:41:35 +0000
+++ curtin/commands/install.py 2017-09-12 21:16:54 +0000
@@ -479,11 +479,23 @@
479 copy_install_log(logfile, workingd.target, log_target_path)479 copy_install_log(logfile, workingd.target, log_target_path)
480 # unmount everything (including iscsi disks)480 # unmount everything (including iscsi disks)
481 util.do_umount(workingd.target, recursive=True)481 util.do_umount(workingd.target, recursive=True)
482 # disconnect configured iscsi disks482
483 LOG.info("Disconnecting iscsi targets (if present)")483 # The open-iscsi service in the ephemeral environment handles
484 for iscsi_disk in iscsi.get_iscsi_disks_from_config(cfg):484 # disconnecting active sessions. On Artful release the systemd
485 LOG.info("Attempting to disconnect %s", iscsi_disk)485 # unit file has conditionals that are not met at boot time and
486 iscsi_disk.disconnect()486 # results in open-iscsi service not being started; This breaks
487 # shutdown on Artful releases.
488 # Additionally, in release < Artful, if the storage configuration
489 # is layered, like RAID over iscsi volumes, then disconnecting iscsi
490 # sessions before stopping the raid device hangs.
491 # As it turns out, letting the open-iscsi service take down the
492 # session last is the cleanest way to handle all releases regardless
493 # of what may be layered on top of the iscsi disks.
494 #
495 # Check if storage configuration has iscsi volumes and if so ensure
496 # iscsi service is active before exiting install
497 if iscsi.get_iscsi_disks_from_config(cfg):
498 iscsi.restart_iscsi_service()
487499
488 shutil.rmtree(workingd.top)500 shutil.rmtree(workingd.top)
489501
490502
=== modified file 'curtin/util.py'
--- curtin/util.py 2017-06-26 16:48:30 +0000
+++ curtin/util.py 2017-09-12 21:16:54 +0000
@@ -58,6 +58,7 @@
58_INSTALLED_MAIN = '/usr/bin/curtin'58_INSTALLED_MAIN = '/usr/bin/curtin'
5959
60_LSB_RELEASE = {}60_LSB_RELEASE = {}
61_USES_SYSTEMD = None
61_HAS_UNSHARE_PID = None62_HAS_UNSHARE_PID = None
6263
63_DNS_REDIRECT_IP = None64_DNS_REDIRECT_IP = None
@@ -1400,4 +1401,17 @@
1400 return data1401 return data
14011402
14021403
1404def uses_systemd():
1405 """ Check if current enviroment uses systemd by testing if
1406 /run/systemd/system is a directory; only present if
1407 systemd is available on running system.
1408 """
1409
1410 global _USES_SYSTEMD
1411 if _USES_SYSTEMD is None:
1412 _USES_SYSTEMD = os.path.isdir('/run/systemd/system')
1413
1414 return _USES_SYSTEMD
1415
1416
1403# vi: ts=4 expandtab syntax=python1417# vi: ts=4 expandtab syntax=python
14041418
=== modified file 'tests/unittests/test_util.py'
--- tests/unittests/test_util.py 2017-08-03 19:31:49 +0000
+++ tests/unittests/test_util.py 2017-09-12 21:16:54 +0000
@@ -851,4 +851,55 @@
851 }, observed)851 }, observed)
852852
853853
854class TestUsesSystemd(CiTestCase):
855
856 def setUp(self):
857 super(TestUsesSystemd, self).setUp()
858 self._reset_cache()
859 self.add_patch('curtin.util.os.path.isdir', 'mock_isdir')
860
861 def _reset_cache(self):
862 util._USES_SYSTEMD = None
863
864 def test_uses_systemd_on_systemd(self):
865 """ Test that uses_systemd returns True if sdpath is a dir """
866 # systemd_enabled
867 self.mock_isdir.return_value = True
868 result = util.uses_systemd()
869 self.assertEqual(True, result)
870 self.assertEqual(1, len(self.mock_isdir.call_args_list))
871
872 def test_uses_systemd_cached(self):
873 """Test that we cache the uses_systemd result"""
874
875 # reset_cache should ensure it's unset
876 self.assertEqual(None, util._USES_SYSTEMD)
877
878 # systemd enabled
879 self.mock_isdir.return_value = True
880
881 # first time
882 first_result = util.uses_systemd()
883
884 # check the cache value
885 self.assertEqual(first_result, util._USES_SYSTEMD)
886
887 # second time
888 second_result = util.uses_systemd()
889
890 # results should match between tries
891 self.assertEqual(True, first_result)
892 self.assertEqual(True, second_result)
893
894 # isdir should only be called once
895 self.assertEqual(1, len(self.mock_isdir.call_args_list))
896
897 def test_uses_systemd_on_non_systemd(self):
898 """ Test that uses_systemd returns False if sdpath is not a dir """
899 # systemd not available
900 self.mock_isdir.return_value = False
901 result = util.uses_systemd()
902 self.assertEqual(False, result)
903
904
854# vi: ts=4 expandtab syntax=python905# vi: ts=4 expandtab syntax=python
855906
=== modified file 'tests/vmtests/test_mdadm_iscsi.py'
--- tests/vmtests/test_mdadm_iscsi.py 2017-08-02 15:46:35 +0000
+++ tests/vmtests/test_mdadm_iscsi.py 2017-09-12 21:16:54 +0000
@@ -36,3 +36,7 @@
3636
37class ZestyTestIscsiMdadm(relbase.zesty, TestMdadmIscsiAbs):37class ZestyTestIscsiMdadm(relbase.zesty, TestMdadmIscsiAbs):
38 __test__ = True38 __test__ = True
39
40
41class ArtfulTestIscsiMdadm(relbase.artful, TestMdadmIscsiAbs):
42 __test__ = True

Subscribers

People subscribed via source and target branches