Merge ~simpoir/ubuntu/+source/landscape-client:beyond_19.04_cosmic into ubuntu/+source/landscape-client:ubuntu/cosmic-devel

Proposed by Simon Poirier
Status: Approved
Approved by: Andreas Hasenack
Approved revision: 599fce02079e26c0ece30d96659b8fc0ba3edf1b
Proposed branch: ~simpoir/ubuntu/+source/landscape-client:beyond_19.04_cosmic
Merge into: ubuntu/+source/landscape-client:ubuntu/cosmic-devel
Diff against target: 377 lines (+326/-1)
6 files modified
debian/changelog (+12/-0)
debian/landscape-client.init (+2/-1)
debian/landscape-client.postinst (+3/-0)
debian/patches/product-name-vminfo-1828217.patch (+49/-0)
debian/patches/series (+2/-0)
debian/patches/stagger-launch-1788518.patch (+258/-0)
Reviewer Review Type Date Requested Status
Andreas Hasenack Approve
Ubuntu Server Dev import team Pending
Review via email: mp+369458@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Same changes as in eoan:

$ git range-diff pkg/ubuntu/cosmic-devel..beyond_19.04_cosmic pkg/import/18.01-0ubuntu8..pkg/ubuntu/devel
1: 289bc1e = 1: 3afe8de Add d/p/product-name-vminfo-1828217.patch
2: 9505ce7 = 2: 9692f3d Default data_path in postinst (LP: #1728681)
3: 70db4fd = 3: e325c2d Add d/p/stagger-launch-1788518.patch
4: 993a71e = 4: 8e14c04 Fix regression with d/landscape-client.init
5: 599fce0 ! 5: ba2b91d Update changelog

- d/changelog version ok for sru

+1

review: Approve
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Sponsoring 599fce02079e26c0ece30d96659b8fc0ba3edf1b

$ git push pkg upload/18.01-0ubuntu4.3
Enumerating objects: 32, done.
Counting objects: 100% (32/32), done.
Delta compression using up to 2 threads
Compressing objects: 100% (25/25), done.
Writing objects: 100% (25/25), 6.97 KiB | 648.00 KiB/s, done.
Total 25 (delta 17), reused 0 (delta 0)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/landscape-client
 * [new tag] upload/18.01-0ubuntu4.3 -> upload/18.01-0ubuntu4.3

$ dput ubuntu ../landscape-client_18.01-0ubuntu4.3_source.changes
Checking signature on .changes
gpg: ../landscape-client_18.01-0ubuntu4.3_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../landscape-client_18.01-0ubuntu4.3.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading landscape-client_18.01-0ubuntu4.3.dsc: done.
  Uploading landscape-client_18.01-0ubuntu4.3.debian.tar.xz: done.
  Uploading landscape-client_18.01-0ubuntu4.3_source.buildinfo: done.
  Uploading landscape-client_18.01-0ubuntu4.3_source.changes: done.
Successfully uploaded packages.

Revision history for this message
Eduardo (eddy89) wrote :

> Same changes as in eoan:
>
> $ git range-diff pkg/ubuntu/cosmic-devel..beyond_19.04_cosmic
> pkg/import/18.01-0ubuntu8..pkg/ubuntu/devel
> 1: 289bc1e = 1: 3afe8de Add d/p/product-name-vminfo-1828217.patch
> 2: 9505ce7 = 2: 9692f3d Default data_path in postinst (LP: #1728681)
> 3: 70db4fd = 3: e325c2d Add d/p/stagger-launch-1788518.patch
> 4: 993a71e = 4: 8e14c04 Fix regression with d/landscape-client.init
> 5: 599fce0 ! 5: ba2b91d Update changelog
>
> - d/changelog version ok for sru
>
> +1

Unmerged commits

599fce0... by Simon Poirier

Update changelog

  * d/p/product-name-vminfo-1828217.patch: Add product_name to things scanned
    for vm_info (LP: #1828217)
  * d/landscape-client.postinst: Set default value if data_path is
    missing. (LP: #1728681)
  * d/p/stagger-launch-1788518.patch: Add option to stagger launch of broker
    plugins. (LP: #1788518)
  * d/landscape-client.init: Fix init script stop action.

993a71e... by Simon Poirier

Fix regression with d/landscape-client.init

Add an additional --name filter to start-stop-daemon to correct
a regression introduced by dpkg>=1.19.3.

70db4fd... by Simon Poirier

Add d/p/stagger-launch-1788518.patch

9505ce7... by Simon Poirier

Default data_path in postinst (LP: #1728681)

289bc1e... by Simon Poirier

Add d/p/product-name-vminfo-1828217.patch

3e79416... by Simon Poirier

Import patches-unapplied version 18.01-0ubuntu4.2 to ubuntu/cosmic-proposed

Imported using git-ubuntu import.

Upload parent: ed8746b0cc500f457aac8debda9328d61c4b33b4

ed8746b... by Simon Poirier

Add debian/patches/mem-usage-1685885.patch:

Reduce memory usage by unused twisted logs. (LP: #1685885)

c4e0207... by Simon Poirier

Import patches-unapplied version 18.01-0ubuntu4.1 to ubuntu/cosmic-proposed

Imported using git-ubuntu import.

Upload parent: f74c019f215ade397dfd9c5ead31b7a6cc5089ce

f74c019... by Simon Poirier

add bug for d/p/nutanix-kvm.patch

1f89b05... by Simon Poirier

Update changelog

  * debian/patches/nutanix-kvm.patch: Update vm_info.py to include Nutanix
    hypervisor.
  * Fixes for release-upgrade.
    - debian/patches/release-upgrade-success.patch: Enable landscape-client to
      survive trusty upgrade.
    - debian/patches/post-upgrade-reboot.patch: Force reboot operation in
      case systemd fails. (LP: #1670291)
  * debian/patches/unicode-tags-script.patch: Permit environments
    containing unicode chars for script execution. (LP: #1765518)
  * debian/patches/1616116-resync-loop.patch:
    Clear hash id database on package resync. (LP: #1616116)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/debian/changelog b/debian/changelog
index 7a04f0d..ad80b2b 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,15 @@
1landscape-client (18.01-0ubuntu4.3) cosmic; urgency=medium
2
3 * d/p/product-name-vminfo-1828217.patch: Add product_name to things scanned
4 for vm_info (LP: #1828217)
5 * d/landscape-client.postinst: Set default value if data_path is
6 missing. (LP: #1728681)
7 * d/p/stagger-launch-1788518.patch: Add option to stagger launch of broker
8 plugins. (LP: #1788518)
9 * d/landscape-client.init: Fix init script stop action (LP: #1833137)
10
11 -- Simon Poirier <simon.poirier@canonical.com> Fri, 28 Jun 2019 10:16:47 -0400
12
1landscape-client (18.01-0ubuntu4.2) cosmic; urgency=medium13landscape-client (18.01-0ubuntu4.2) cosmic; urgency=medium
214
3 * debian/patches/mem-usage-1685885.patch: Reduce memory usage by15 * debian/patches/mem-usage-1685885.patch: Reduce memory usage by
diff --git a/debian/landscape-client.init b/debian/landscape-client.init
index fcaeafd..ed51cc2 100644
--- a/debian/landscape-client.init
+++ b/debian/landscape-client.init
@@ -63,7 +63,8 @@ case "$1" in
6363
64 stop)64 stop)
65 log_daemon_msg "Stopping $NAME daemon"65 log_daemon_msg "Stopping $NAME daemon"
66 start-stop-daemon --retry 30 --stop --quiet --pidfile $PIDFILE66 # not a typo, linux process names are truncated to 15chars
67 start-stop-daemon --retry 30 --stop --quiet --pidfile $PIDFILE -n landscape-clien
67 log_end_msg $?68 log_end_msg $?
68 ;;69 ;;
6970
diff --git a/debian/landscape-client.postinst b/debian/landscape-client.postinst
index d8d91ba..6e441ea 100644
--- a/debian/landscape-client.postinst
+++ b/debian/landscape-client.postinst
@@ -102,6 +102,9 @@ END
102 # user information. The flag file will be removed by the client when102 # user information. The flag file will be removed by the client when
103 # the update completes.103 # the update completes.
104 DATA_PATH="`grep ^data_path /etc/landscape/client.conf | cut -d= -f2 | tr -d '[[:space:]]'`"104 DATA_PATH="`grep ^data_path /etc/landscape/client.conf | cut -d= -f2 | tr -d '[[:space:]]'`"
105 if [ -z "$DATA_PATH" ]; then
106 DATA_PATH=/var/lib/landscape/client
107 fi
105 install --owner=landscape --directory $DATA_PATH108 install --owner=landscape --directory $DATA_PATH
106 USER_UPDATE_FLAG_FILE="$DATA_PATH/user-update-flag"109 USER_UPDATE_FLAG_FILE="$DATA_PATH/user-update-flag"
107 install --owner=landscape /dev/null $USER_UPDATE_FLAG_FILE110 install --owner=landscape /dev/null $USER_UPDATE_FLAG_FILE
diff --git a/debian/patches/product-name-vminfo-1828217.patch b/debian/patches/product-name-vminfo-1828217.patch
108new file mode 100644111new file mode 100644
index 0000000..4420e0c
--- /dev/null
+++ b/debian/patches/product-name-vminfo-1828217.patch
@@ -0,0 +1,49 @@
1Description: Add product_name to things scanned for vm_info
2Author: Simon Poirier <simon.poirier@canonical.com>
3Origin: upstream, https://github.com/CanonicalLtd/landscape-client/commit/ad28feeb7d3031268eeb206c4d8bdcd015ad79ae
4Bug-Ubuntu: https://bugs.launchpad.net/bugs/1828217
5Last-Update: 2019-06-14
6
7--- a/landscape/lib/tests/test_vm_info.py
8+++ b/landscape/lib/tests/test_vm_info.py
9@@ -179,6 +179,11 @@
10 self.make_dmi_info("sys_vendor", "Some other vendor")
11 self.assertEqual(b"", get_vm_info(root_path=self.root_path))
12
13+ def test_get_vm_info_with_kvm_product(self):
14+ """get_vm_info returns 'kvm', if product_name is 'KVM'."""
15+ self.make_dmi_info("product_name", "KVM")
16+ self.assertEqual(b"kvm", get_vm_info(root_path=self.root_path))
17+
18
19 class GetContainerInfoTest(BaseTestCase):
20
21--- a/landscape/lib/vm_info.py
22+++ b/landscape/lib/vm_info.py
23@@ -6,6 +6,9 @@
24 from landscape.lib.fs import read_text_file
25
26
27+DMI_FILES = ("sys_vendor", "chassis_vendor", "bios_vendor", "product_name")
28+
29+
30 def get_vm_info(root_path="/"):
31 """
32 Return a bytestring with the virtualization type if it's known, an empty
33@@ -22,7 +25,7 @@
34 # Iterate through all dmi *_vendors, as clouds can (and will) customize
35 # sysinfo values. (https://libvirt.org/formatdomain.html#elementsSysinfo)
36 dmi_info_path = os.path.join(root_path, "sys/class/dmi/id")
37- for dmi_info_file in ("sys_vendor", "chassis_vendor", "bios_vendor"):
38+ for dmi_info_file in DMI_FILES:
39 dmi_vendor_path = os.path.join(dmi_info_path, dmi_info_file)
40 if not os.path.exists(dmi_vendor_path):
41 continue
42@@ -76,6 +79,7 @@
43 ("nutanix", b"kvm"),
44 ("openstack", b"kvm"),
45 ("qemu", b"kvm"),
46+ ("kvm", b"kvm"),
47 ("vmware", b"vmware"))
48 for name, vm_type in content_vendors_map:
49 if name in vendor:
diff --git a/debian/patches/series b/debian/patches/series
index 79d72dd..ebfbf13 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -7,3 +7,5 @@ post-upgrade-reboot.patch
7unicode-tags-script.patch7unicode-tags-script.patch
81616116-resync-loop.patch81616116-resync-loop.patch
9mem-usage-1685885.patch9mem-usage-1685885.patch
10product-name-vminfo-1828217.patch
11stagger-launch-1788518.patch
diff --git a/debian/patches/stagger-launch-1788518.patch b/debian/patches/stagger-launch-1788518.patch
10new file mode 10064412new file mode 100644
index 0000000..d0f2908
--- /dev/null
+++ b/debian/patches/stagger-launch-1788518.patch
@@ -0,0 +1,258 @@
1Description: Add option to stagger launch of broker plugins.
2Author: Simon Poirier <simon.poirier@canonical.com>
3Origin: upstream, https://github.com/CanonicalLtd/landscape-client/commit/597a584083f5245b8960e6c20d67e0fe25bdcdc2
4Bug-Ubuntu: https://bugs.launchpad.net/bugs/1788518
5Last-Update: 2019-06-14
6
7diff --git a/example.conf b/example.conf
8index 47dac275..3965d29d 100644
9--- a/example.conf
10+++ b/example.conf
11@@ -50,6 +50,16 @@ ping_url = http://landscape.canonical.com/ping
12 #
13 #ssl_public_key
14
15+# The ratio over which to spread landscape intervals. This is to mitigate
16+# the impact of restarting the host of multiple landscape-client instances.
17+# Values range from 0.0 to 1.0, from unstaggered to spreading over
18+# the full interval.
19+#
20+# Default value is 0.1, meaning jobs will start with up to 10% delay
21+#
22+# Example:
23+# stagger_launch = 0.5
24+
25 # If set to True interrupt (SIGINT) signals will be ignored by the
26 # landscape-client daemon.
27 ignore_sigint = False
28diff --git a/landscape/client/broker/client.py b/landscape/client/broker/client.py
29index 62287329..042c18a4 100644
30--- a/landscape/client/broker/client.py
31+++ b/landscape/client/broker/client.py
32@@ -1,5 +1,6 @@
33-from logging import info, exception, error
34+from logging import info, exception, error, debug
35 import sys
36+import random
37
38 from twisted.internet.defer import maybeDeferred, succeed
39
40@@ -105,9 +106,18 @@ class BrokerClientPlugin(object):
41 if self.run_immediately:
42 self._run_with_error_log()
43 if self.run_interval is not None:
44- self._loop = self.client.reactor.call_every(
45- self.run_interval,
46- self._run_with_error_log)
47+ delay = (random.random() * self.run_interval *
48+ self.client.config.stagger_launch)
49+ debug("delaying start of %s for %d seconds",
50+ format_object(self), delay)
51+ self._loop = self.client.reactor.call_later(
52+ delay, self._start_loop)
53+
54+ def _start_loop(self):
55+ """Launch the client loop."""
56+ self._loop = self.client.reactor.call_every(
57+ self.run_interval,
58+ self._run_with_error_log)
59
60 def _run_with_error_log(self):
61 """Wrap self.run in a Deferred with a logging error handler."""
62@@ -140,10 +150,11 @@ class BrokerClient(object):
63 """
64 name = "client"
65
66- def __init__(self, reactor):
67+ def __init__(self, reactor, config):
68 super(BrokerClient, self).__init__()
69 self.reactor = reactor
70 self.broker = None
71+ self.config = config
72 self._registered_messages = {}
73 self._plugins = []
74 self._plugin_names = {}
75diff --git a/landscape/client/broker/tests/helpers.py b/landscape/client/broker/tests/helpers.py
76index 97c5e4f4..fc010209 100644
77--- a/landscape/client/broker/tests/helpers.py
78+++ b/landscape/client/broker/tests/helpers.py
79@@ -206,7 +206,9 @@ class BrokerClientHelper(RemoteBrokerHelper):
80 # The client needs its own reactor to avoid infinite loops
81 # when the broker broadcasts and event
82 test_case.client_reactor = FakeReactor()
83- test_case.client = BrokerClient(test_case.client_reactor)
84+ config = BrokerConfiguration()
85+ config.stagger_launch = 0 # let's keep tests deterministic
86+ test_case.client = BrokerClient(test_case.client_reactor, config)
87 test_case.client.broker = test_case.remote
88
89
90diff --git a/landscape/client/broker/tests/test_client.py b/landscape/client/broker/tests/test_client.py
91index 967f77db..3295b02b 100644
92--- a/landscape/client/broker/tests/test_client.py
93+++ b/landscape/client/broker/tests/test_client.py
94@@ -178,6 +178,26 @@ class BrokerClientTest(LandscapeTest):
95 deferred.callback(123)
96 self.assertEquals(runs, [True, True])
97
98+ @mock.patch("random.random")
99+ def test_run_interval_staggered(self, mock_random):
100+ """
101+ If a plugin has a run method and staggered_launch is set,
102+ the launch gets delayed by a random factor.
103+ """
104+ mock_random.return_value = 1.0
105+ plugin = BrokerClientPlugin()
106+ plugin.run_interval = 60
107+ plugin.run = mock.Mock()
108+ self.client.config.stagger_launch = 0.5
109+ self.client.add(plugin)
110+ self.client_reactor.advance(30)
111+ self.assertEqual(0, plugin.run.call_count)
112+ self.client_reactor.advance(60)
113+ self.assertEqual(1, plugin.run.call_count)
114+ self.client_reactor.advance(60)
115+ self.assertEqual(2, plugin.run.call_count)
116+ self.assertEqual(1, mock_random.call_count)
117+
118 def test_run_immediately(self):
119 """
120 If a plugin has a C{run} method and C{run_immediately} is C{True},
121diff --git a/landscape/client/deployment.py b/landscape/client/deployment.py
122index 8c2f905e..7c5464cb 100644
123--- a/landscape/client/deployment.py
124+++ b/landscape/client/deployment.py
125@@ -82,6 +82,7 @@ class Configuration(BaseConfiguration):
126 - C{ping_url} (C{"http://landscape.canonical.com/ping"})
127 - C{ssl_public_key}
128 - C{ignore_sigint} (C{False})
129+ - C{stagger_launch} (C{0.1})
130 """
131 parser = super(Configuration, self).make_parser()
132 logging.add_cli_options(parser, logdir="/var/log/landscape")
133@@ -111,6 +112,10 @@ class Configuration(BaseConfiguration):
134 metavar="INTERVAL",
135 help="The number of seconds between flushes to disk "
136 "for persistent data.")
137+ parser.add_option("--stagger-launch", metavar="STAGGER_RATIO",
138+ dest="stagger_launch", default=0.1, type=float,
139+ help="Ratio, between 0 and 1, by which to scatter "
140+ "various tasks of landscape.")
141
142 # Hidden options, used for load-testing to run in-process clones
143 parser.add_option("--clones", default=0, type=int, help=SUPPRESS_HELP)
144diff --git a/landscape/client/manager/manager.py b/landscape/client/manager/manager.py
145index a662e13e..18d13148 100644
146--- a/landscape/client/manager/manager.py
147+++ b/landscape/client/manager/manager.py
148@@ -12,7 +12,7 @@ class Manager(BrokerClient):
149 name = "manager"
150
151 def __init__(self, reactor, config):
152- super(Manager, self).__init__(reactor)
153+ super(Manager, self).__init__(reactor, config)
154 self.reactor = reactor
155 self.config = config
156 self.store = ManagerStore(self.config.store_filename)
157diff --git a/landscape/client/monitor/monitor.py b/landscape/client/monitor/monitor.py
158index 7ab82560..ebd3823d 100644
159--- a/landscape/client/monitor/monitor.py
160+++ b/landscape/client/monitor/monitor.py
161@@ -12,7 +12,7 @@ class Monitor(BrokerClient):
162
163 def __init__(self, reactor, config, persist, persist_filename=None,
164 step_size=5 * 60):
165- super(Monitor, self).__init__(reactor)
166+ super(Monitor, self).__init__(reactor, config)
167 self.reactor = reactor
168 self.config = config
169 self.persist = persist
170diff --git a/landscape/client/tests/helpers.py b/landscape/client/tests/helpers.py
171index 73775f99..a15353fc 100644
172--- a/landscape/client/tests/helpers.py
173+++ b/landscape/client/tests/helpers.py
174@@ -175,6 +175,7 @@ class MonitorHelper(FakeBrokerServiceHelper):
175 persist_filename = test_case.makePersistFile()
176 test_case.config = MonitorConfiguration()
177 test_case.config.load(["-c", test_case.config_filename])
178+ test_case.config.stagger_launch = 0 # let's keep tests deterministic
179 test_case.reactor = FakeReactor()
180 test_case.monitor = Monitor(
181 test_case.reactor, test_case.config,
182diff --git a/landscape/client/tests/test_configuration.py b/landscape/client/tests/test_configuration.py
183index 58381321..36d6d3ec 100644
184--- a/landscape/client/tests/test_configuration.py
185+++ b/landscape/client/tests/test_configuration.py
186@@ -753,6 +753,13 @@ class LandscapeSetupScriptTest(LandscapeTest):
187 self.assertEqual(self.config.access_group, u"webservers")
188 mock_input.assert_not_called()
189
190+ @mock.patch("landscape.client.configuration.input")
191+ def test_stagger_defined_on_command_line(self, mock_input):
192+ self.assertEqual(self.config.stagger_launch, 0.1)
193+ self.config.load_command_line(["--stagger-launch", "0.5"])
194+ self.assertEqual(self.config.stagger_launch, 0.5)
195+ mock_input.assert_not_called()
196+
197 @mock.patch("landscape.client.configuration.show_help")
198 def test_show_header(self, mock_show_help):
199 help_snippet = "This script will"
200diff --git a/man/landscape-client.1 b/man/landscape-client.1
201index 3a16c4a9..adb4bdc1 100644
202--- a/man/landscape-client.1
203+++ b/man/landscape-client.1
204@@ -1,5 +1,5 @@
205 .\" Text automatically generated by txt2man
206-.TH landscape-client 1 "05 January 2017" "" ""
207+.TH landscape-client 1 "14 February 2019" "" ""
208 .SH NAME
209 \fBlandscape-client \fP- Landscape system client
210 \fB
211diff --git a/man/landscape-config.1 b/man/landscape-config.1
212index 6f362f2f..7d398fff 100644
213--- a/man/landscape-config.1
214+++ b/man/landscape-config.1
215@@ -1,5 +1,5 @@
216 .\" Text automatically generated by txt2man
217-.TH landscape-config 1 "05 January 2017" "" ""
218+.TH landscape-config 1 "14 February 2019" "" ""
219 .SH NAME
220 \fBlandscape-config \fP- configure the Landscape management client
221 \fB
222@@ -139,6 +139,11 @@ Comma separated list of tag names to be sent to the
223 server.
224 .TP
225 .B
226+\fB--stagger-launch\fP=STAGGER_RATIO
227+Ratio, between 0 and 1, by which to scatter various
228+tasks of landscape.
229+.TP
230+.B
231 \fB--import\fP=FILENAME_OR_URL
232 Filename or URL to import configuration from.
233 Imported \fIoptions\fP behave as if they were passed in
234diff --git a/man/landscape-config.txt b/man/landscape-config.txt
235index 223398b7..5a100bf0 100644
236--- a/man/landscape-config.txt
237+++ b/man/landscape-config.txt
238@@ -58,6 +58,8 @@ OPTIONS
239 EC2 or UEC. Read below for details.
240 --tags=TAGS Comma separated list of tag names to be sent to the
241 server.
242+ --stagger-launch=STAGGER_RATIO Ratio, between 0 and 1, by which to scatter various
243+ tasks of landscape.
244 --import=FILENAME_OR_URL Filename or URL to import configuration from.
245 Imported options behave as if they were passed in
246 the command line, with precedence being given to
247diff --git a/man/landscape-sysinfo.1 b/man/landscape-sysinfo.1
248index e8dd2ff4..90852975 100644
249--- a/man/landscape-sysinfo.1
250+++ b/man/landscape-sysinfo.1
251@@ -1,5 +1,5 @@
252 .\" Text automatically generated by txt2man
253-.TH landscape-sysinfo 1 "07 March 2018" "" ""
254+.TH landscape-sysinfo 1 "14 February 2019" "" ""
255 .SH NAME
256 \fBlandscape-sysinfo \fP- Display a summary of the current system status
257 \fB
258

Subscribers

People subscribed via source and target branches