Merge ~mitchburton/ubuntu/+source/landscape-client:ubuntu/lunar-devel into ubuntu/+source/landscape-client:ubuntu/lunar-devel

Proposed by Mitch Burton
Status: Merged
Merge reported by: Robie Basak
Merged at revision: e14dec00a3918bec7c87b50d271fec1cb95bd4a9
Proposed branch: ~mitchburton/ubuntu/+source/landscape-client:ubuntu/lunar-devel
Merge into: ubuntu/+source/landscape-client:ubuntu/lunar-devel
Diff against target: 4985 lines (+1840/-569)
89 files modified
.github/workflows/ci.yml (+34/-0)
Makefile (+6/-8)
Makefile.packaging (+1/-1)
README (+58/-15)
debian/changelog (+29/-0)
debian/control (+1/-1)
debian/landscape-client.logrotate (+2/-1)
debian/landscape-client.postrm (+3/-0)
debian/patches/series (+0/-7)
debian/rules (+1/-1)
dev/null (+0/-25)
example.conf (+8/-0)
landscape/__init__.py (+1/-1)
landscape/client/broker/exchange.py (+57/-9)
landscape/client/broker/registration.py (+18/-13)
landscape/client/broker/server.py (+9/-0)
landscape/client/broker/service.py (+2/-1)
landscape/client/broker/tests/badprivate.ssl (+25/-13)
landscape/client/broker/tests/badpublic.ssl (+19/-14)
landscape/client/broker/tests/helpers.py (+10/-8)
landscape/client/broker/tests/private.ssl (+25/-13)
landscape/client/broker/tests/public.ssl (+19/-14)
landscape/client/broker/tests/test_exchange.py (+184/-0)
landscape/client/broker/tests/test_registration.py (+46/-17)
landscape/client/broker/tests/test_server.py (+19/-0)
landscape/client/broker/tests/test_store.py (+1/-1)
landscape/client/broker/tests/test_transport.py (+1/-1)
landscape/client/broker/transport.py (+1/-1)
landscape/client/configuration.py (+43/-2)
landscape/client/lockfile.py (+52/-0)
landscape/client/manager/aptsources.py (+6/-8)
landscape/client/manager/config.py (+8/-0)
landscape/client/manager/keystonetoken.py (+1/-1)
landscape/client/manager/scriptexecution.py (+3/-5)
landscape/client/manager/service.py (+2/-1)
landscape/client/manager/tests/test_aptsources.py (+10/-198)
landscape/client/manager/tests/test_processkiller.py (+1/-1)
landscape/client/manager/tests/test_scriptexecution.py (+13/-13)
landscape/client/monitor/__init__.py (+1/-1)
landscape/client/monitor/computertags.py (+29/-0)
landscape/client/monitor/config.py (+1/-1)
landscape/client/monitor/processorinfo.py (+42/-2)
landscape/client/monitor/service.py (+2/-1)
landscape/client/monitor/tests/test_computertags.py (+64/-0)
landscape/client/monitor/tests/test_ubuntuproinfo.py (+47/-0)
landscape/client/monitor/ubuntuproinfo.py (+47/-0)
landscape/client/package/changer.py (+2/-2)
landscape/client/package/reporter.py (+5/-0)
landscape/client/package/tests/test_changer.py (+3/-3)
landscape/client/package/tests/test_releaseupgrader.py (+2/-2)
landscape/client/package/tests/test_reporter.py (+26/-0)
landscape/client/package/tests/test_taskhandler.py (+4/-1)
landscape/client/reactor.py (+3/-0)
landscape/client/tests/test_amp.py (+89/-1)
landscape/client/tests/test_configuration.py (+73/-2)
landscape/client/tests/test_lockfile.py (+21/-0)
landscape/client/user/provider.py (+1/-1)
landscape/lib/apt/package/facade.py (+19/-6)
landscape/lib/apt/package/skeleton.py (+1/-2)
landscape/lib/apt/package/testing.py (+5/-5)
landscape/lib/apt/package/tests/test_facade.py (+67/-0)
landscape/lib/apt/package/tests/test_skeleton.py (+10/-2)
landscape/lib/backoff.py (+53/-0)
landscape/lib/base64.py (+8/-0)
landscape/lib/bpickle.py (+1/-1)
landscape/lib/compat.py (+5/-1)
landscape/lib/config.py (+1/-1)
landscape/lib/disk.py (+2/-2)
landscape/lib/gpg.py (+17/-5)
landscape/lib/lsb_release.py (+48/-15)
landscape/lib/message.py (+14/-4)
landscape/lib/network.py (+99/-66)
landscape/lib/schema.py (+18/-4)
landscape/lib/testing.py (+1/-1)
landscape/lib/tests/test_backoff.py (+37/-0)
landscape/lib/tests/test_config.py (+15/-6)
landscape/lib/tests/test_gpg.py (+43/-4)
landscape/lib/tests/test_lsb_release.py (+45/-4)
landscape/lib/tests/test_network.py (+93/-12)
landscape/lib/tests/test_schema.py (+2/-2)
landscape/lib/tests/test_vm_info.py (+15/-3)
landscape/lib/user.py (+1/-1)
landscape/lib/vm_info.py (+2/-3)
landscape/message_schemas/server_bound.py (+14/-4)
landscape/message_schemas/test_message.py (+9/-1)
landscape/sysinfo/load.py (+2/-1)
landscape/sysinfo/network.py (+2/-1)
man/landscape-config.1 (+7/-1)
man/landscape-config.txt (+3/-0)
Reviewer Review Type Date Requested Status
Andreas Hasenack Approve
Ubuntu Sponsors Pending
git-ubuntu import Pending
Review via email: mp+437634@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Mitch Burton (mitchburton) wrote :

PPA: https://launchpad.net/~mitchburton/+archive/ubuntu/landscape-client-ppa

We have been following the development procedure laid out in https://wiki.ubuntu.com/LandscapeUpdates

Furthermore, the testing procedure results can be seen in
https://wiki.canonical.com/Landscape/ClientSRUTests/23.02

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Looking at this now.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

This branch doesn't build:
$ dquilt push -a
Applying patch 0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch
patching file landscape/lib/network.py
Hunk #1 FAILED at 248.
1 out of 1 hunk FAILED -- rejects in file landscape/lib/network.py
Patch 0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch does not apply (enforce with -f)

I grabbed the source package from your ppa:
$ dget https://launchpad.net/~mitchburton/+archive/ubuntu/landscape-client-ppa/+sourcefiles/landscape-client/23.02-0ubuntu1/landscape-client_23.02-0ubuntu1.dsc
(...)

And that source package doesn't have a debian/patches directory.

You have a lot of patches still being applied from debian/patches/series:
$ cat debian/patches/series
0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch
0002-lp1870087-stale-locks.patch
py3.9.patch
0003-clean-publisher-shutdown.patch
replace-tostring.patch
1962539_twisted_py3.patch
lp1903776-release-upgrade.patch

review: Needs Fixing
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

If I stop applying all the patches from this branch (debian/patches/series list), and try to build the package using the 23.03 orig source tarball from your ppa, I get one discrepancy:

--- landscape-client-23.02.orig/landscape/lib/disk.py
+++ landscape-client-23.02/landscape/lib/disk.py
@@ -10,7 +10,7 @@ from landscape.lib.compat import _PY3
 # List of filesystem types authorized when generating disk use statistics.
 STABLE_FILESYSTEMS = frozenset(
     ["ext", "ext2", "ext3", "ext4", "reiserfs", "ntfs", "msdos", "dos", "vfat",
- "xfs", "hpfs", "jfs", "ufs", "hfs", "hfsplus", "simfs", "drvfs", "lxfs"])
+ "xfs", "hpfs", "jfs", "ufs", "hfs", "hfsplus", "simfs"])

That is a diff between the orig tarball, and the branch contents. They should be identical.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

While that is sorted out, I'll try to check the other changes, starting with debian/* (the packaging)

Revision history for this message
Mitch Burton (mitchburton) wrote :

Thanks. Looks like I missed a commit when sorting out the changes from the last version (adding to the STABLE_FILESYSTEMS set). I'm currently going through the patches to make sure they were applied upstream and sorting out the ones that were.

Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Is this commit closing bug #1878957?

commit a8759e3f79b13f1ae7dc864ae15f6064af2829d0
Author: Simon Poirier <email address hidden>
Date: Fri Jun 5 11:52:32 2020 -0400

    Preserve auto flag on package changer operations. (#85)

    address issues with package not autoremoved (LP: #1878957).

Maybe mention it in d/changelog?

I don´t recall the landscape client case/exception 100%, but keep in mind every bug mentioned in d/changelog might need to have an SRU template (test case, regression analysis, etc). Or how have you been dealing with that in past SRUs?

Revision history for this message
Mitch Burton (mitchburton) :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Here is another case:
commit 35a35f3b59aed70242688a6f3c87e584fac5fc0c
Author: Simon Poirier <email address hidden>
Date: Tue Aug 11 18:42:24 2020 -0400

    decode registration error codes as str (#87)

    fixes (LP: #1889464)

This bug at least does not have an ubuntu task, just a landscape-client (project) one. I suppose maybe back then LC bugs were tracked in LP instead of github?

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

This one is still "in progress" in LP (in the landscape-client project, not ubuntu packages), and I didn't find it mentioned in any previous d/changelog entry. Checking the code it's still an open issue in the current lunar package, which this branch is fixing:

commit 77a8b93e082e3df82213c9f3973898ec1c36e3fc
Author: Simon Poirier <email address hidden>
Date: Mon Nov 16 09:48:09 2020 -0500

    Add RHEV to vm_info map. (LP: #1884116) (#86)

Are these all old bugs from before the move to github? Or is the project also tracking bugs in lp? I'm unsure now if they all should be listed in d/changelog, as there are going to be many, looks like.

Revision history for this message
Kevin Nasto (silverdrake11) :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Another one still "in progress" in LP:

commit 4e518be77cdd1f755d6d86cbfc71ea558aa11308
Author: Simon Poirier <email address hidden>
Date: Fri Sep 24 15:35:19 2021 -0400

    Translate legacy py27 messages to py3 messages in broker server. (#94)

    This should address issues with upgrades never reporting back
    when migrating to python3. (LP: #1943291)

Revision history for this message
Mitch Burton (mitchburton) wrote :

Right. There was a transition from launchpad to github, but bugs should still be solely tracked in launchpad. I'll add the missing ones to d/changelog (and update the bug report statuses).

Past SRUs have been a bit of a mixed bag, with some older ones I've seen covering multiple bugs in a single template.

The landscape client case/exception, at least as detailed at the start of https://wiki.ubuntu.com/LandscapeUpdates, is simply that "new features are allowed in an update" as long as the QA process is observed.

Revision history for this message
Mitch Burton (mitchburton) :
Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Mitch Burton (mitchburton) :
Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Mitch Burton (mitchburton) :
Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Andreas Hasenack (ahasenack) :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I made the d/control debhelper changes and sponsored.

Things to discuss/do after this upload, before the SRU potentially:
- revisit debhelper for the SRU, and even lunar
- handle the case where ua is not installed, or not executable. We shouldn't spam the LS server every 15min with an error in that case
- gpg key handling: use a more specific filename for the keyfiles, and restrict the globbing in postrm accordingly

e14dec0... by Mitch Burton

Set debhelper-compat version to 12 for lunar.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

+1

review: Approve
Revision history for this message
Robie Basak (racb) wrote :

Seb asked me to mark this as Merged as it was uploaded in https://launchpad.net/ubuntu/+source/landscape-client/23.02-0ubuntu1

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
2new file mode 100644
3index 0000000..54fd142
4--- /dev/null
5+++ b/.github/workflows/ci.yml
6@@ -0,0 +1,34 @@
7+name: ci
8+on: [pull_request, workflow_dispatch]
9+jobs:
10+ check3:
11+ runs-on: ${{ matrix.os }}
12+ strategy:
13+ matrix:
14+ os: ["ubuntu-22.04", "ubuntu-20.04"]
15+ steps:
16+ - uses: actions/checkout@v2
17+ - run: |
18+ make depends
19+ # -common seems a catch-22, but this is just a shortcut to
20+ # initialize user and dirs, some used through tests.
21+ sudo apt-get -y install landscape-common
22+ - run: make check3 TRIAL=/usr/bin/trial3
23+ lint:
24+ runs-on: ubuntu-latest
25+ steps:
26+ - uses: actions/checkout@v2
27+ - run: make depends
28+ - run: make lint
29+ coverage:
30+ runs-on: ubuntu-latest
31+ steps:
32+ - uses: actions/checkout@v2
33+ - run: |
34+ make depends
35+ # -common seems a catch-22, but this is just a shortcut to
36+ # initialize user and dirs, some used through tests.
37+ sudo apt-get -y install landscape-common
38+ - run: make coverage TRIAL=/usr/bin/trial3
39+ - name: upload
40+ uses: codecov/codecov-action@v1
41diff --git a/.travis.yml b/.travis.yml
42deleted file mode 100644
43index b0fed89..0000000
44--- a/.travis.yml
45+++ /dev/null
46@@ -1,71 +0,0 @@
47-dist: trusty
48-sudo: true
49-language: python
50-matrix:
51- include:
52- - env: TARGET=check2 IMAGE=ubuntu:xenial
53- script:
54- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo apt-get update && sudo apt-get -y install make python-distutils-extra python-mock python-twisted python-apt python-twisted-core python-pycurl python-netifaces"';
55- # creates user and dirs, some used through tests
56- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo apt-get -y install landscape-common"';
57- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo chown -R ubuntu:ubuntu /target"';
58- - sg lxd -c 'lxc exec testcontainer -- sudo -i -u ubuntu sh -c "cd /target; make ${TARGET}" ubuntu';
59- - env: TARGET=check3 IMAGE=ubuntu:bionic
60- script:
61- # bionic needs cgroupv2 not on this kernel, so nudge the system a bit
62- - sg lxd -c 'lxc exec testcontainer -- sh -c "dhclient eth0; cloud-init init"';
63- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo apt-get update && sudo apt-get -y install make python3-distutils-extra python3-mock python3-twisted python3-apt python-twisted-core python3-pycurl python3-netifaces"';
64- # creates user and dirs, some used through tests
65- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo apt-get -y install landscape-common"';
66- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo chown -R ubuntu:ubuntu /target"';
67- - sg lxd -c 'lxc exec testcontainer -- sudo -i -u ubuntu sh -c "cd /target; make ${TARGET}" ubuntu';
68- - env: TARGET=check3 IMAGE=ubuntu-daily:focal
69- script:
70- # bionic needs cgroupv2 not on this kernel, so nudge the system a bit
71- - sg lxd -c 'lxc exec testcontainer -- sh -c "dhclient eth0; cloud-init init"';
72- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo apt-get update && sudo apt-get -y install make python3-distutils-extra python3-mock python3-twisted python3-apt python-twisted-core python3-pycurl python3-netifaces net-tools"';
73- # creates user and dirs, some used through tests
74- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo apt-get -y install landscape-common"';
75- - sg lxd -c 'lxc exec testcontainer -- sh -c "sudo chown -R ubuntu:ubuntu /target"';
76- - sg lxd -c 'lxc exec testcontainer -- sudo -i -u ubuntu sh -c "cd /target; make ${TARGET}" ubuntu';
77- - python: 3.6
78- env: TARGET=lint
79- install:
80- - pip install flake8
81- - python: 3.4
82- env: TARGET=coverage
83- before_script:
84- - python3 -m pip install -U coverage
85- - python3 -m pip install -U codecov
86- install:
87- - pip install flake8
88- # These match "make depends"
89- - pip install twisted==16.4.0 mock==1.3.0 configobj==5.0.6 pycurl netifaces==0.10.4
90- - pip install http://launchpad.net/python-distutils-extra/trunk/2.39/+download/python-distutils-extra-2.39.tar.gz
91- # build & install python-apt
92- - make -f Makefile.travis pipinstallpythonapt
93- script:
94- - make $TARGET
95- after_success:
96- - codecov
97-env:
98- global:
99- - TRIAL_ARGS=-j4
100-before_script:
101- - if [ ! -z ${IMAGE} ]; then
102- sudo apt-get -y -t trusty-backports install lxd;
103- sudo lxd init --auto;
104- sudo usermod -a -G lxd travis;
105- sudo sed -e 's/LXD_IPV4_ADDR=""/LXD_IPV4_ADDR="10.0.8.1"/' -e 's/LXD_IPV4_NETMASK=""/LXD_IPV4_NETMASK="255.255.255.0"/' -e 's:LXD_IPV4_NETWORK="":LXD_IPV4_NETWORK="10.0.8.0/24":' -e 's/LXD_IPV4_DHCP_RANGE=""/LXD_IPV4_DHCP_RANGE="10.0.8.2,10.0.8.254"/' -e 's/LXD_IPV4_DHCP_MAX=""/LXD_IPV4_DHCP_MAX="250"/' -i /etc/default/lxd-bridge;
106- sudo /etc/init.d/lxd restart;
107- fi
108- - if [ ! -z ${IMAGE} ]; then
109- sg lxd -c 'lxc launch ${IMAGE} testcontainer -c security.privileged=true';
110- echo "waiting for nic";
111- for t in 1 2 5 8; do sg lxd -c 'lxc list' | grep 10.0.8 && break; sleep ${t}; done;
112- sg lxd -c 'lxc config device add testcontainer srcdir disk path=/target source=${PWD}';
113- sg lxd -c 'lxc config device add testcontainer srcdir disk path=/target source=${PWD}';
114- sg lxd -c 'lxc exec testcontainer -- sh -c "echo 1 | sudo tee /proc/sys/net/ipv6/conf/all/disable_ipv6"';
115- fi
116-script:
117- - make $TARGET PYTHON=python$TRAVIS_PYTHON_VERSION;
118diff --git a/Makefile b/Makefile
119index fb49996..fe183ec 100644
120--- a/Makefile
121+++ b/Makefile
122@@ -2,7 +2,7 @@ PYDOCTOR ?= pydoctor
123 TXT2MAN ?= txt2man
124 PYTHON2 ?= python2
125 PYTHON3 ?= python3
126-TRIAL ?= $(shell which trial)
127+TRIAL ?= -m twisted.trial
128 TRIAL_ARGS ?=
129
130 .PHONY: help
131@@ -10,16 +10,16 @@ help: ## Print help about available targets
132 @grep -h -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'
133
134 .PHONY: depends
135-depends: depends2 depends3 ## Install py2 and py3 dependencies.
136- sudo apt -y install python3-flake8 python3-coverage
137+depends: depends3 ## py2 is deprecated
138+ sudo apt-get -y install python3-flake8 python3-coverage
139
140 .PHONY: depends2
141 depends2:
142- sudo apt -y install python-twisted-core python-distutils-extra python-mock python-configobj python-netifaces
143+ sudo apt-get -y install python-twisted-core python-distutils-extra python-mock python-configobj python-netifaces python-pycurl
144
145 .PHONY: depends3
146 depends3:
147- sudo apt -y install python3-twisted python3-distutils-extra python3-mock python3-configobj python3-netifaces
148+ sudo apt-get -y install python3-twisted python3-distutils-extra python3-mock python3-configobj python3-netifaces python3-pycurl
149
150 all: build
151
152@@ -52,9 +52,7 @@ check3: build3
153 .PHONY: coverage
154 coverage:
155 PYTHONPATH=$(PYTHONPATH):$(CURDIR) LC_ALL=C $(PYTHON3) -m coverage run $(TRIAL) --unclean-warnings landscape
156-
157-.PHONY: ci-check
158-ci-check: depends build check ## Install dependencies and run all the tests.
159+ PYTHONPATH=$(PYTHONPATH):$(CURDIR) LC_ALL=C $(PYTHON3) -m coverage xml
160
161 .PHONY: lint
162 lint:
163diff --git a/Makefile.packaging b/Makefile.packaging
164index 068eb5f..5a43ce7 100644
165--- a/Makefile.packaging
166+++ b/Makefile.packaging
167@@ -3,7 +3,7 @@ UBUNTU_RELEASE := $(shell lsb_release -cs)
168 # Use := here, not =, it's really important, otherwise UPSTREAM_VERSION
169 # will be updated behind your back with the current result of that
170 # command everytime it is mentioned/used.
171-UPSTREAM_VERSION := $(shell python -c "from landscape import UPSTREAM_VERSION; print(UPSTREAM_VERSION)")
172+UPSTREAM_VERSION := $(shell python3 -c "from landscape import UPSTREAM_VERSION; print(UPSTREAM_VERSION)")
173 CHANGELOG_VERSION := $(shell dpkg-parsechangelog | grep ^Version | cut -f 2 -d " " | cut -f 1 -d '-')
174 GIT_HASH := $(shell git rev-parse --short HEAD)
175 # We simulate a git "revno" for the sake of sortability.
176diff --git a/Makefile.travis b/Makefile.travis
177deleted file mode 100644
178index b532241..0000000
179--- a/Makefile.travis
180+++ /dev/null
181@@ -1,40 +0,0 @@
182-UBUNTU_RELEASE := $(shell lsb_release -cs)
183-
184-# We'd use the packaged version, but travis runs python in virtualenv,
185-# thus this pip workaround.
186-.PHONY: pipinstallpythonapt
187-pipinstallpythonapt: pipinstallpythonapt_deps
188- $(MAKE) -f $(lastword $(MAKEFILE_LIST)) pipinstallpythonapt_default || \
189- $(MAKE) -f $(lastword $(MAKEFILE_LIST)) pipinstallpythonapt_src_$(UBUNTU_RELEASE)
190-
191-.PHONY: pipinstallpythonapt_deps
192-pipinstallpythonapt_deps:
193- pip install pyopenssl
194- pip install service_identity
195- sudo apt-get update
196- sudo apt-get -y build-dep python-apt python3-apt
197- sudo apt-get -y install libapt-pkg-dev
198-
199-.PHONY: pipinstallpythonapt_default
200-pipinstallpythonapt_default:
201- # See: https://code.launchpad.net/ubuntu/+source/python-apt
202- bzr branch lp:ubuntu/$(UBUNTU_RELEASE)/python-apt /tmp/python-apt
203- pip install /tmp/python-apt
204-
205-.PHONY: pipinstallpythonapt_src
206-pipinstallpythonapt_src:
207- wget -O /tmp/python-apt_$(PYAPT_VER).tar.xz https://launchpad.net/ubuntu/+archive/primary/+files/python-apt_$(PYAPT_VER).tar.xz
208- # pip2.7 does not support .xz
209- xz -dfk /tmp/python-apt_$(PYAPT_VER).tar.xz
210- pip install /tmp/python-apt_$(PYAPT_VER).tar
211-
212-.PHONY: pipinstallpythonapt_src_xenial
213-pipinstallpythonapt_src_xenial: PYAPT_VER = 1.1.0~beta1build1
214-pipinstallpythonapt_src_xenial: pipinstallpythonapt_src
215-
216-# travis-ci nodes use a backported version of apt incompatible with
217-# the version of python-apt on trusty.
218-# This version matches the libapt-pkg on travis nodes so it compiles.
219-.PHONY: pipinstallpythonapt_src_trusty
220-pipinstallpythonapt_src_trusty: PYAPT_VER = 1.1.0~beta1build1
221-pipinstallpythonapt_src_trusty: pipinstallpythonapt_src
222diff --git a/README b/README
223index 8e5e4b1..06be0e1 100644
224--- a/README
225+++ b/README
226@@ -1,45 +1,88 @@
227-[![Build Status](https://travis-ci.org/CanonicalLtd/landscape-client.svg?branch=master)](https://travis-ci.org/CanonicalLtd/landscape-client)
228+[![Build Status](https://github.com/CanonicalLtd/landscape-client/actions/workflows/ci.yml/badge.svg)](https://github.com/CanonicalLtd/landscape-client/actions/workflows/ci.yml)
229 [![codecov](https://codecov.io/gh/CanonicalLtd/landscape-client/branch/master/graph/badge.svg)](https://codecov.io/gh/CanonicalLtd/landscape-client)
230
231-== Non-root mode ==
232+## Installation Instructions
233
234-The Landscape Client generally runs as a combination of the 'root' and
235-'landscape' users. It is possible to disable the administrative features of
236-Landscape and run only the monitoring parts of it without using the 'root'
237+Add our beta PPA to get the latest updates to the landscape-client package
238+
239+#### Add repo to an Ubuntu series
240+```
241+sudo add-apt-repository ppa:landscape/self-hosted-beta
242+```
243+
244+#### Add repo to a Debian based series that is not Ubuntu (experimental)
245+
246+```
247+# 1. Install our signing key
248+gpg --keyserver keyserver.ubuntu.com --recv-keys 6e85a86e4652b4e6
249+gpg --export 6e85a86e4652b4e6 | sudo tee -a /usr/share/keyrings/landscape-client-keyring.gpg > /dev/null
250+
251+# 2. Add repository
252+echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/landscape-client-keyring.gpg] https://ppa.launchpadcontent.net/landscape/self-hosted-beta/ubuntu focal main" | sudo tee -a /etc/apt/sources.list.d/landscape-client.list
253+```
254+
255+#### Install the package
256+```
257+sudo apt update && sudo apt install landscape-client
258+```
259+
260+## Non-root mode
261+
262+The Landscape Client generally runs as a combination of the `root` and
263+`landscape` users. It is possible to disable the administrative features of
264+Landscape and run only the monitoring parts of it without using the `root`
265 user at all.
266
267 If you wish to use the Landscape Client in this way, it's recommended that you
268 perform these steps immediately after installing the landscape-client package.
269
270-Edit /etc/default/landscape-client and add the following lines:
271+Edit `/etc/default/landscape-client` and add the following lines:
272
273- RUN=1
274- DAEMON_USER=landscape
275+```
276+RUN=1
277+DAEMON_USER=landscape
278+```
279
280-Edit /etc/landscape/client.conf and add the following line:
281+Edit `/etc/landscape/client.conf` and add the following line:
282+```
283+monitor_only = true
284+```
285
286- monitor_only = true
287+## Running
288
289-Now you can run 'sudo landscape-config' as usual to complete the configuration
290-of your client and register with the Landscape service.
291+Now you can complete the configuration of your client and register with the
292+Landscape service. There are two ways to do this:
293
294+1. `sudo landscape-config` and answer interactive prompts to finalize your configuration
295+2. `sudo landscape-config --account-name standalone --url https://<server>/message-system --ping-url http://<server>/ping` if registering to a self-hosted Landscape instance. Replace `<server>` with the hostname of your self-hosted Landscape instance.
296
297-== Developing ==
298+## Developing
299
300 To run the full test suite, run the following command:
301
302+```
303 make check
304+```
305
306 When you want to test the landscape client manually without management
307 features, you can simply run:
308
309+```
310 $ ./scripts/landscape-client
311+```
312
313-This defaults to the 'landscape-client.conf' configuration file.
314+This defaults to the `landscape-client.conf` configuration file.
315
316 When you want to test management features manually, you'll need to run as root.
317-There's a configuration file 'root-client.conf' which specifies use of the
318+There's a configuration file `root-client.conf` which specifies use of the
319 system bus.
320
321+```
322 $ sudo ./scripts/landscape-client -c root-client.conf
323+```
324
325+Before opening a PR, make sure to run the full testsuite and lint
326+```
327+make check3
328+make lint
329+```
330diff --git a/debian/changelog b/debian/changelog
331index fcf9dca..4ba984f 100644
332--- a/debian/changelog
333+++ b/debian/changelog
334@@ -1,3 +1,32 @@
335+landscape-client (23.02-0ubuntu1) lunar; urgency=medium
336+
337+ * New upstream release 23.02:
338+ - Preventing the generation of large messages and logs that can overwhelm
339+ Landscape Server (LP: #1995775)
340+ - Improved MOTD slowdown on machines with many tap network interfaces
341+ (LP: #2006396)
342+ - No longer using deprecated apt-key when storing trusted GPG keys
343+ (LP: #1973202)
344+ - Fixed issue recognising Parallels VMs as Virtual Machine clients
345+ (LP: #1827909)
346+ - Fixes for incorrect logfile rotation config (LP: #1968189)
347+ - Client-side backoff handling to moderate traffic to Landscape Server
348+ during high load (LP: #1947399)
349+ - Avoid sending empty messages when catching up to expected next message
350+ (LP: #1917540)
351+ - --is-registered CLI option to quickly check if client is registered
352+ (LP: #1912516)
353+ - Can now report Ubuntu Pro attachment information if the version of
354+ Landscape Server it is registered to supports this (LP: #2006401)
355+ - Packages installed as dependencies as part of package profiles are now
356+ appropriately autoremovable (LP: #1878957)
357+ - Registration timeouts give an error instead of timing out (LP: #1889464)
358+ - RHEV hypervisor VMs are now recognized as virtual machines (LP: #1884116)
359+ - Doing a Landscape-driven release upgrade from a release running python 2
360+ to one running python 3 no longer hangs forever (LP: #1943291)
361+
362+ -- Mitch Burton <mitch.burton@canonical.com> Wed, 08 Feb 2023 10:23:31 -0800
363+
364 landscape-client (19.12-0ubuntu13) jammy; urgency=medium
365
366 * d/landscape-sysinfo.wrapper, d/landscape-common.postrm: avoid too
367diff --git a/debian/compat b/debian/compat
368deleted file mode 100644
369index 48082f7..0000000
370--- a/debian/compat
371+++ /dev/null
372@@ -1 +0,0 @@
373-12
374diff --git a/debian/control b/debian/control
375index e6178c8..3c8af12 100644
376--- a/debian/control
377+++ b/debian/control
378@@ -3,7 +3,7 @@ Section: admin
379 Priority: optional
380 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
381 XSBC-Original-Maintainer: Landscape Team <landscape-team@canonical.com>
382-Build-Depends: debhelper (>= 12), po-debconf, libdistro-info-perl,
383+Build-Depends: debhelper (= 12), po-debconf, libdistro-info-perl,
384 dh-python, python3-dev, python3-distutils-extra,
385 lsb-release, gawk, net-tools,
386 python3-apt, python3-twisted, python3-configobj
387diff --git a/debian/landscape-client.logrotate b/debian/landscape-client.logrotate
388index c91ebf0..09f86e2 100644
389--- a/debian/landscape-client.logrotate
390+++ b/debian/landscape-client.logrotate
391@@ -5,8 +5,9 @@
392 notifempty
393 compress
394 nocreate
395+ sharedscripts
396 postrotate
397- [ -f /var/run/landscape/landscape-client.pid ] && kill -s USR1 `cat /var/run/landscape/landscape-client.pid` > /dev/null 2>&1 || :
398+ systemctl kill --signal=USR1 --kill-who=main landscape-client > /dev/null 2>&1 || :
399 endscript
400 }
401
402diff --git a/debian/landscape-client.postrm b/debian/landscape-client.postrm
403index a436e39..10f79b8 100644
404--- a/debian/landscape-client.postrm
405+++ b/debian/landscape-client.postrm
406@@ -18,6 +18,7 @@ case "$1" in
407 purge)
408 LOG_DIR=/var/log/landscape
409 DATA_DIR=/var/lib/landscape
410+ GPG_DIR=/etc/apt/trusted.gpg.d
411
412 rm -f "/etc/default/landscape-client"
413
414@@ -28,6 +29,8 @@ case "$1" in
415 rm -f "${LOG_DIR}/package-reporter.log"*
416 rm -f "${LOG_DIR}/package-changer.log"*
417
418+ rm -f "${GPG_DIR}/landscape-server"*.asc
419+
420 rm -rf "${DATA_DIR}/client"
421 rm -rf "${DATA_DIR}/.gnupg"
422
423diff --git a/debian/patches/0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch b/debian/patches/0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch
424deleted file mode 100644
425index 49e8b44..0000000
426--- a/debian/patches/0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch
427+++ /dev/null
428@@ -1,28 +0,0 @@
429-From 97f74e0354d60b78a75fd7ed96c3f7fad6cd35ef Mon Sep 17 00:00:00 2001
430-From: Balint Reczey <balint.reczey@canonical.com>
431-Date: Sat, 7 Dec 2019 10:11:09 +0100
432-Subject: [PATCH] Handle EINVAL error of SIOCETHTOOL ioctl
433-
434-The error occurs on WSL (1) and causes an error showing up in MOTD.
435-
436-LP: #1855522
437----
438- landscape/lib/network.py | 4 ++--
439- 1 file changed, 2 insertions(+), 2 deletions(-)
440-
441---- a/landscape/lib/network.py
442-+++ b/landscape/lib/network.py
443-@@ -248,11 +248,11 @@
444- fcntl.ioctl(sock, SIOCETHTOOL, packed) # Status ioctl() call
445- res = status_cmd.tostring()
446- speed, duplex = struct.unpack("12xHB28x", res)
447-- except IOError as e:
448-+ except (IOError, OSError) as e:
449- if e.errno == errno.EPERM:
450- logging.warn("Could not determine network interface speed, "
451- "operation not permitted.")
452-- elif e.errno != errno.EOPNOTSUPP:
453-+ elif e.errno != errno.EOPNOTSUPP and e.errno != errno.EINVAL:
454- raise e
455- speed = -1
456- duplex = False
457diff --git a/debian/patches/0002-lp1870087-stale-locks.patch b/debian/patches/0002-lp1870087-stale-locks.patch
458deleted file mode 100644
459index f57b5b6..0000000
460--- a/debian/patches/0002-lp1870087-stale-locks.patch
461+++ /dev/null
462@@ -1,210 +0,0 @@
463-Description: Clean up stale lock files
464- Rework lockfile to look for process names.
465-Author: Simon Poirier <simon.poirier@canonical.com>
466-Origin: upstream, https://github.com/CanonicalLtd/landscape-client/commit/2450eacd331097431122b9861613b8a03e5f74d9
467-Bug-Ubuntu: https://bugs.launchpad.net/bugs/1870087
468-
469-Index: landscape-client-19.12/landscape/client/lockfile.py
470-===================================================================
471---- /dev/null
472-+++ landscape-client-19.12/landscape/client/lockfile.py
473-@@ -0,0 +1,52 @@
474-+import os
475-+
476-+from twisted.python import lockfile
477-+
478-+
479-+def patch_lockfile():
480-+ if lockfile.FilesystemLock is PatchedFilesystemLock:
481-+ return
482-+ lockfile.FilesystemLock = PatchedFilesystemLock
483-+
484-+
485-+class PatchedFilesystemLock(lockfile.FilesystemLock):
486-+ """
487-+ Patched Twisted's FilesystemLock.lock to handle PermissionError
488-+ when trying to lock.
489-+ """
490-+
491-+ def lock(self):
492-+ # XXX Twisted assumes PIDs don't get reused, which is incorrect.
493-+ # As such, we pre-check that any existing lock file isn't
494-+ # associated to a live process, and that any associated
495-+ # process is from landscape. Otherwise, clean up the lock file,
496-+ # considering it to be locked to a recycled PID.
497-+ #
498-+ # Although looking for the process name may seem fragile, it's the
499-+ # most acurate info we have since:
500-+ # * some process run as root, so the UID is not a reference
501-+ # * process may not be spawned by systemd, so cgroups are not reliable
502-+ # * python executable is not a reference
503-+ clean = True
504-+ try:
505-+ pid = os.readlink(self.name)
506-+ ps_name = get_process_name(int(pid))
507-+ if not ps_name.startswith("landscape"):
508-+ os.remove(self.name)
509-+ clean = False
510-+ except Exception:
511-+ # We can't figure the lock state, let FilesystemLock figure it
512-+ # out normally.
513-+ pass
514-+
515-+ result = super(PatchedFilesystemLock, self).lock()
516-+ self.clean = self.clean and clean
517-+ return result
518-+
519-+
520-+def get_process_name(pid):
521-+ """Return a process name from a pid."""
522-+ stat_path = "/proc/{}/stat".format(pid)
523-+ with open(stat_path) as stat_file:
524-+ stat = stat_file.read()
525-+ return stat.partition("(")[2].rpartition(")")[0]
526-Index: landscape-client-19.12/landscape/client/reactor.py
527-===================================================================
528---- landscape-client-19.12.orig/landscape/client/reactor.py
529-+++ landscape-client-19.12/landscape/client/reactor.py
530-@@ -2,6 +2,9 @@
531- Extend the regular Twisted reactor with event-handling features.
532- """
533- from landscape.lib.reactor import EventHandlingReactor
534-+from landscape.client.lockfile import patch_lockfile
535-+
536-+patch_lockfile()
537-
538-
539- class LandscapeReactor(EventHandlingReactor):
540-Index: landscape-client-19.12/landscape/client/tests/test_amp.py
541-===================================================================
542---- landscape-client-19.12.orig/landscape/client/tests/test_amp.py
543-+++ landscape-client-19.12/landscape/client/tests/test_amp.py
544-@@ -1,9 +1,17 @@
545--from twisted.internet.error import ConnectError
546-+import os
547-+import errno
548-+import subprocess
549-+import textwrap
550-+
551-+import mock
552-+
553-+from twisted.internet.error import ConnectError, CannotListenError
554- from twisted.internet.task import Clock
555-
556- from landscape.client.tests.helpers import LandscapeTest
557- from landscape.client.deployment import Configuration
558- from landscape.client.amp import ComponentPublisher, ComponentConnector, remote
559-+from landscape.client.reactor import LandscapeReactor
560- from landscape.lib.amp import MethodCallError
561- from landscape.lib.testing import FakeReactor
562-
563-@@ -159,3 +167,83 @@ class ComponentConnectorTest(LandscapeTe
564- effectively a no-op.
565- """
566- self.connector.disconnect()
567-+
568-+ @mock.patch("twisted.python.lockfile.kill")
569-+ def test_stale_locks_with_dead_pid(self, mock_kill):
570-+ """Publisher starts with stale lock."""
571-+ mock_kill.side_effect = [
572-+ OSError(errno.ESRCH, "No such process")]
573-+ sock_path = os.path.join(self.config.sockets_path, u"test.sock")
574-+ lock_path = u"{}.lock".format(sock_path)
575-+ # fake a PID which does not exist
576-+ os.symlink("-1", lock_path)
577-+
578-+ component = TestComponent()
579-+ # Test the actual Unix reactor implementation. Fakes won't do.
580-+ reactor = LandscapeReactor()
581-+ publisher = ComponentPublisher(component, reactor, self.config)
582-+
583-+ # Shouldn't raise the exception.
584-+ publisher.start()
585-+
586-+ # ensure stale lock was replaced
587-+ self.assertNotEqual("-1", os.readlink(lock_path))
588-+ mock_kill.assert_called_with(-1, 0)
589-+
590-+ publisher.stop()
591-+ reactor._cleanup()
592-+
593-+ @mock.patch("twisted.python.lockfile.kill")
594-+ def test_stale_locks_recycled_pid(self, mock_kill):
595-+ """Publisher starts with stale lock pointing to recycled process."""
596-+ mock_kill.side_effect = [
597-+ OSError(errno.EPERM, "Operation not permitted")]
598-+ sock_path = os.path.join(self.config.sockets_path, u"test.sock")
599-+ lock_path = u"{}.lock".format(sock_path)
600-+ # fake a PID recycled by a known process which isn't landscape (init)
601-+ os.symlink("1", lock_path)
602-+
603-+ component = TestComponent()
604-+ # Test the actual Unix reactor implementation. Fakes won't do.
605-+ reactor = LandscapeReactor()
606-+ publisher = ComponentPublisher(component, reactor, self.config)
607-+
608-+ # Shouldn't raise the exception.
609-+ publisher.start()
610-+
611-+ # ensure stale lock was replaced
612-+ self.assertNotEqual("1", os.readlink(lock_path))
613-+ mock_kill.assert_not_called()
614-+ self.assertFalse(publisher._port.lockFile.clean)
615-+
616-+ publisher.stop()
617-+ reactor._cleanup()
618-+
619-+ @mock.patch("twisted.python.lockfile.kill")
620-+ def test_with_valid_lock(self, mock_kill):
621-+ """Publisher raises lock error if a valid lock is held."""
622-+ sock_path = os.path.join(self.config.sockets_path, u"test.sock")
623-+ lock_path = u"{}.lock".format(sock_path)
624-+ # fake a landscape process
625-+ app = self.makeFile(textwrap.dedent("""\
626-+ #!/usr/bin/python3
627-+ import time
628-+ time.sleep(10)
629-+ """), basename="landscape-manager")
630-+ os.chmod(app, 0o755)
631-+ call = subprocess.Popen([app])
632-+ self.addCleanup(call.terminate)
633-+ os.symlink(str(call.pid), lock_path)
634-+
635-+ component = TestComponent()
636-+ # Test the actual Unix reactor implementation. Fakes won't do.
637-+ reactor = LandscapeReactor()
638-+ publisher = ComponentPublisher(component, reactor, self.config)
639-+
640-+ with self.assertRaises(CannotListenError):
641-+ publisher.start()
642-+
643-+ # ensure lock was not replaced
644-+ self.assertEqual(str(call.pid), os.readlink(lock_path))
645-+ mock_kill.assert_called_with(call.pid, 0)
646-+ reactor._cleanup()
647-Index: landscape-client-19.12/landscape/client/tests/test_lockfile.py
648-===================================================================
649---- /dev/null
650-+++ landscape-client-19.12/landscape/client/tests/test_lockfile.py
651-@@ -0,0 +1,21 @@
652-+import os
653-+import subprocess
654-+import textwrap
655-+
656-+from landscape.client import lockfile
657-+from landscape.client.tests.helpers import LandscapeTest
658-+
659-+
660-+class LockFileTest(LandscapeTest):
661-+
662-+ def test_read_process_name(self):
663-+ app = self.makeFile(textwrap.dedent("""\
664-+ #!/usr/bin/python3
665-+ import time
666-+ time.sleep(10)
667-+ """), basename="my_fancy_app")
668-+ os.chmod(app, 0o755)
669-+ call = subprocess.Popen([app])
670-+ self.addCleanup(call.terminate)
671-+ proc_name = lockfile.get_process_name(call.pid)
672-+ self.assertEqual("my_fancy_app", proc_name)
673diff --git a/debian/patches/0003-clean-publisher-shutdown.patch b/debian/patches/0003-clean-publisher-shutdown.patch
674deleted file mode 100644
675index 3c7bc1e..0000000
676--- a/debian/patches/0003-clean-publisher-shutdown.patch
677+++ /dev/null
678@@ -1,49 +0,0 @@
679-Description: Return missing deferred on service shutdown.
680- This should allow services to let their publisher shut down and clean
681- after itself, thus avoiding stale locks and sockets.
682-Author: Simon Poirier <simon.poirier@canonical.com>
683-Origin: upstream, https://github.com/CanonicalLtd/landscape-client/commit/8a617a1aafbbe9d972410707563710c3cbedd8e7
684-Bug-Ubuntu: https://bugs.launchpad.net/bugs/1870087
685---- a/landscape/client/broker/service.py
686-+++ b/landscape/client/broker/service.py
687-@@ -82,10 +82,11 @@
688-
689- def stopService(self):
690- """Stop the broker."""
691-- self.publisher.stop()
692-+ deferred = self.publisher.stop()
693- self.exchanger.stop()
694- self.pinger.stop()
695- super(BrokerService, self).stopService()
696-+ return deferred
697-
698-
699- def run(args):
700---- a/landscape/client/manager/service.py
701-+++ b/landscape/client/manager/service.py
702-@@ -54,8 +54,9 @@
703- def stopService(self):
704- """Stop the manager and close the connection with the broker."""
705- self.connector.disconnect()
706-- self.publisher.stop()
707-+ deferred = self.publisher.stop()
708- super(ManagerService, self).stopService()
709-+ return deferred
710-
711-
712- def run(args):
713---- a/landscape/client/monitor/service.py
714-+++ b/landscape/client/monitor/service.py
715-@@ -56,10 +56,11 @@
716- The monitor is flushed to ensure that things like persist databases
717- get saved to disk.
718- """
719-- self.publisher.stop()
720-+ deferred = self.publisher.stop()
721- self.monitor.flush()
722- self.connector.disconnect()
723- super(MonitorService, self).stopService()
724-+ return deferred
725-
726-
727- def run(args):
728diff --git a/debian/patches/1962539_twisted_py3.patch b/debian/patches/1962539_twisted_py3.patch
729deleted file mode 100644
730index 7593ffd..0000000
731--- a/debian/patches/1962539_twisted_py3.patch
732+++ /dev/null
733@@ -1,156 +0,0 @@
734-Description: Fix obsolete imports on jammy.
735- * replace deprecated twisted _PY3
736- * replace legacy types which are gone (unicode, long)
737-Author: Simon Poirier <simon.poirier@canonical.com>
738-Origin: backport, https://github.com/CanonicalLtd/landscape-client/commit/80d3da45ca8773d3eee262d4de5dc6e6b8d99121
739-Bug-Ubuntu: https://bugs.launchpad.net/bugs/1962539
740-Last-Update: 2022-03-03
741-
742---- a/landscape/client/broker/exchange.py
743-+++ b/landscape/client/broker/exchange.py
744-@@ -347,7 +347,7 @@
745- from landscape.lib.hashlib import md5
746-
747- from twisted.internet.defer import Deferred, succeed
748--from twisted.python.compat import _PY3
749-+from landscape.lib.compat import _PY3
750-
751- from landscape.lib.fetch import HTTPCodeError, PyCurlError
752- from landscape.lib.format import format_delta
753---- a/landscape/client/broker/tests/test_registration.py
754-+++ b/landscape/client/broker/tests/test_registration.py
755-@@ -3,7 +3,7 @@
756- import socket
757- import mock
758-
759--from twisted.python.compat import _PY3
760-+from landscape.lib.compat import _PY3
761-
762- from landscape.client.broker.registration import RegistrationError, Identity
763- from landscape.client.tests.helpers import LandscapeTest
764---- a/landscape/client/broker/transport.py
765-+++ b/landscape/client/broker/transport.py
766-@@ -6,7 +6,7 @@
767-
768- import pycurl
769-
770--from twisted.python.compat import unicode, _PY3
771-+from landscape.lib.compat import unicode, _PY3
772-
773- from landscape.lib import bpickle
774- from landscape.lib.fetch import fetch
775---- a/landscape/client/manager/keystonetoken.py
776-+++ b/landscape/client/manager/keystonetoken.py
777-@@ -1,7 +1,7 @@
778- import os
779- import logging
780-
781--from twisted.python.compat import _PY3
782-+from landscape.lib.compat import _PY3
783-
784- from landscape.lib.compat import ConfigParser, NoOptionError
785- from landscape.client.monitor.plugin import DataWatcher
786---- a/landscape/client/user/provider.py
787-+++ b/landscape/client/user/provider.py
788-@@ -4,7 +4,7 @@
789- import logging
790- import subprocess
791-
792--from twisted.python.compat import _PY3
793-+from landscape.lib.compat import _PY3
794-
795-
796- class UserManagementError(Exception):
797---- a/landscape/lib/apt/package/skeleton.py
798-+++ b/landscape/lib/apt/package/skeleton.py
799-@@ -1,9 +1,8 @@
800-+from landscape.lib.compat import unicode, _PY3
801- from landscape.lib.hashlib import sha1
802-
803- import apt_pkg
804-
805--from twisted.python.compat import unicode, _PY3
806--
807-
808- PACKAGE = 1 << 0
809- PROVIDES = 1 << 1
810---- a/landscape/lib/bpickle.py
811-+++ b/landscape/lib/bpickle.py
812-@@ -32,7 +32,7 @@
813- wire compatible and behave the same way (bugs notwithstanding).
814- """
815-
816--from twisted.python.compat import _PY3
817-+from landscape.lib.compat import long, _PY3
818-
819- dumps_table = {}
820- loads_table = {}
821---- a/landscape/lib/compat.py
822-+++ b/landscape/lib/compat.py
823-@@ -1,6 +1,6 @@
824- # flake8: noqa
825-
826--from twisted.python.compat import _PY3
827-+_PY3 = str != bytes
828-
829-
830- if _PY3:
831-@@ -13,6 +13,8 @@
832- from io import StringIO
833- stringio = cstringio = StringIO
834- from builtins import input
835-+ unicode = str
836-+ long = int
837-
838- else:
839- import cPickle
840-@@ -24,3 +26,5 @@
841- stringio = StringIO
842- from cStringIO import StringIO as cstringio
843- input = raw_input
844-+ long = long
845-+ unicode = unicode
846---- a/landscape/lib/disk.py
847-+++ b/landscape/lib/disk.py
848-@@ -4,7 +4,7 @@
849- import re
850- import codecs
851-
852--from twisted.python.compat import _PY3
853-+from landscape.lib.compat import _PY3
854-
855-
856- # List of filesystem types authorized when generating disk use statistics.
857---- a/landscape/lib/network.py
858-+++ b/landscape/lib/network.py
859-@@ -11,7 +11,7 @@
860- import logging
861-
862- import netifaces
863--from twisted.python.compat import long, _PY3
864-+from landscape.lib.compat import long, _PY3
865-
866- __all__ = ["get_active_device_info", "get_network_traffic"]
867-
868---- a/landscape/lib/testing.py
869-+++ b/landscape/lib/testing.py
870-@@ -15,7 +15,7 @@
871- from logging import Handler, ERROR, Formatter
872- from twisted.trial.unittest import TestCase
873- from twisted.python.compat import StringType as basestring
874--from twisted.python.compat import _PY3
875-+from landscape.lib.compat import _PY3
876- from twisted.python.failure import Failure
877- from twisted.internet.defer import Deferred
878- from twisted.internet.error import ConnectError
879---- a/landscape/lib/user.py
880-+++ b/landscape/lib/user.py
881-@@ -1,7 +1,7 @@
882- import os.path
883- import pwd
884-
885--from twisted.python.compat import _PY3
886-+from landscape.lib.compat import _PY3
887-
888- from landscape.lib.encoding import encode_if_needed
889-
890diff --git a/debian/patches/lp1903776-release-upgrade.patch b/debian/patches/lp1903776-release-upgrade.patch
891deleted file mode 100644
892index bd3a154..0000000
893--- a/debian/patches/lp1903776-release-upgrade.patch
894+++ /dev/null
895@@ -1,141 +0,0 @@
896-Description: Use /etc/apt/trusted.gpg.d for validating upgrade-tool signature.
897-Author: Simon Poirier <simon.poirier@canonical.com>
898-Origin: upstream, https://github.com/CanonicalLtd/landscape-client/commit/bcbe04db6ca45c7a0afac171e85a9c88c19253ae
899-Bug-Ubuntu: https://bugs.launchpad.net/bugs/1903776
900-
901-diff --git a/landscape/lib/gpg.py b/landscape/lib/gpg.py
902-index 66400d18..58218c54 100644
903---- a/landscape/lib/gpg.py
904-+++ b/landscape/lib/gpg.py
905-@@ -1,6 +1,9 @@
906-+import itertools
907- import shutil
908- import tempfile
909-
910-+from glob import glob
911-+
912- from twisted.internet.utils import getProcessOutputAndValue
913-
914-
915-@@ -8,14 +11,15 @@ class InvalidGPGSignature(Exception):
916- """Raised when the gpg signature for a given file is invalid."""
917-
918-
919--def gpg_verify(filename, signature, gpg="/usr/bin/gpg"):
920-+def gpg_verify(filename, signature, gpg="/usr/bin/gpg", apt_dir="/etc/apt"):
921- """Verify the GPG signature of a file.
922-
923- @param filename: Path to the file to verify the signature against.
924- @param signature: Path to signature to use.
925- @param gpg: Optionally, path to the GPG binary to use.
926-+ @param apt_dir: Optionally, path to apt trusted keyring.
927- @return: a C{Deferred} resulting in C{True} if the signature is
928-- valid, C{False} otherwise.
929-+ valid, C{False} otherwise.
930- """
931-
932- def remove_gpg_home(ignored):
933-@@ -32,9 +36,17 @@ def check_gpg_exit_code(args):
934- "code='%d')" % (gpg, out, err, code))
935-
936- gpg_home = tempfile.mkdtemp()
937-- args = ("--no-options", "--homedir", gpg_home, "--no-default-keyring",
938-- "--ignore-time-conflict", "--keyring", "/etc/apt/trusted.gpg",
939-- "--verify", signature, filename)
940-+ keyrings = tuple(itertools.chain(*[
941-+ ("--keyring", keyring)
942-+ for keyring in sorted(
943-+ glob("{}/trusted.gpg".format(apt_dir)) +
944-+ glob("{}/trusted.gpg.d/*.gpg".format(apt_dir))
945-+ )
946-+ ]))
947-+ args = (
948-+ "--no-options", "--homedir", gpg_home, "--no-default-keyring",
949-+ "--ignore-time-conflict"
950-+ ) + keyrings + ("--verify", signature, filename)
951-
952- result = getProcessOutputAndValue(gpg, args=args)
953- result.addBoth(remove_gpg_home)
954-diff --git a/landscape/lib/tests/test_gpg.py b/landscape/lib/tests/test_gpg.py
955-index e6165a26..4c84e008 100644
956---- a/landscape/lib/tests/test_gpg.py
957-+++ b/landscape/lib/tests/test_gpg.py
958-@@ -1,9 +1,10 @@
959- import mock
960- import os
961-+import textwrap
962- import unittest
963-
964- from twisted.internet import reactor
965--from twisted.internet.defer import Deferred
966-+from twisted.internet.defer import Deferred, inlineCallbacks
967-
968- from landscape.lib import testing
969- from landscape.lib.gpg import gpg_verify
970-@@ -16,6 +17,8 @@ def test_gpg_verify(self):
971- L{gpg_verify} runs the given gpg binary and returns C{True} if the
972- provided signature is valid.
973- """
974-+ aptdir = self.makeDir()
975-+ os.mknod("{}/trusted.gpg".format(aptdir))
976- gpg_options = self.makeFile()
977- gpg = self.makeFile("#!/bin/sh\n"
978- "touch $3/trustdb.gpg\n"
979-@@ -27,14 +30,15 @@ def test_gpg_verify(self):
980- @mock.patch("tempfile.mkdtemp")
981- def do_test(mkdtemp_mock):
982- mkdtemp_mock.return_value = gpg_home
983-- result = gpg_verify("/some/file", "/some/signature", gpg=gpg)
984-+ result = gpg_verify(
985-+ "/some/file", "/some/signature", gpg=gpg, apt_dir=aptdir)
986-
987- def check_result(ignored):
988- self.assertEqual(
989- open(gpg_options).read(),
990- "--no-options --homedir %s --no-default-keyring "
991-- "--ignore-time-conflict --keyring /etc/apt/trusted.gpg "
992-- "--verify /some/signature /some/file" % gpg_home)
993-+ "--ignore-time-conflict --keyring %s/trusted.gpg "
994-+ "--verify /some/signature /some/file" % (gpg_home, aptdir))
995- self.assertFalse(os.path.exists(gpg_home))
996-
997- result.addCallback(check_result)
998-@@ -70,3 +74,38 @@ def check_failure(failure):
999-
1000- reactor.callWhenRunning(do_test)
1001- return deferred
1002-+
1003-+ @inlineCallbacks
1004-+ def test_gpg_verify_trusted_dir(self):
1005-+ """
1006-+ gpg_verify uses keys from the trusted.gpg.d if such a folder exists.
1007-+ """
1008-+ apt_dir = self.makeDir()
1009-+ os.mkdir("{}/trusted.gpg.d".format(apt_dir))
1010-+ os.mknod("{}/trusted.gpg.d/foo.gpg".format(apt_dir))
1011-+ os.mknod("{}/trusted.gpg.d/baz.gpg".format(apt_dir))
1012-+ os.mknod("{}/trusted.gpg.d/bad.gpg~".format(apt_dir))
1013-+
1014-+ gpg_call = self.makeFile()
1015-+ fake_gpg = self.makeFile(textwrap.dedent("""\
1016-+ #!/bin/sh
1017-+ touch $3/trustdb.gpg
1018-+ echo -n $@ > {}
1019-+ """).format(gpg_call))
1020-+ os.chmod(fake_gpg, 0o755)
1021-+ gpg_home = self.makeDir()
1022-+
1023-+ with mock.patch("tempfile.mkdtemp", return_value=gpg_home):
1024-+ yield gpg_verify(
1025-+ "/some/file", "/some/signature", gpg=fake_gpg, apt_dir=apt_dir)
1026-+
1027-+ expected = (
1028-+ "--no-options --homedir {gpg_home} --no-default-keyring "
1029-+ "--ignore-time-conflict "
1030-+ "--keyring {apt_dir}/trusted.gpg.d/baz.gpg "
1031-+ "--keyring {apt_dir}/trusted.gpg.d/foo.gpg "
1032-+ "--verify /some/signature /some/file"
1033-+ ).format(gpg_home=gpg_home, apt_dir=apt_dir)
1034-+ with open(gpg_call) as call:
1035-+ self.assertEqual(expected, call.read())
1036-+ self.assertFalse(os.path.exists(gpg_home))
1037diff --git a/debian/patches/py3.9.patch b/debian/patches/py3.9.patch
1038deleted file mode 100644
1039index 4c70c7d..0000000
1040--- a/debian/patches/py3.9.patch
1041+++ /dev/null
1042@@ -1,65 +0,0 @@
1043-Description: Switch from depreated base64.decodestring to
1044- base64.decodebytes to fix FTBFS with python3.9.
1045-Author: Dimitri John Ledkov <xnox@ubuntu.com>
1046-
1047-
1048---- landscape-client-19.12.orig/landscape/client/configuration.py
1049-+++ landscape-client-19.12/landscape/client/configuration.py
1050-@@ -513,7 +513,7 @@ def decode_base64_ssl_public_certificate
1051- # WARNING: ssl_public_certificate is misnamed, it's not the key of the
1052- # certificate, but the actual certificate itself.
1053- if config.ssl_public_key and config.ssl_public_key.startswith("base64:"):
1054-- decoded_cert = base64.decodestring(
1055-+ decoded_cert = base64.decodebytes(
1056- config.ssl_public_key[7:].encode("ascii"))
1057- config.ssl_public_key = store_public_key_data(
1058- config, decoded_cert)
1059---- landscape-client-19.12.orig/landscape/client/package/changer.py
1060-+++ landscape-client-19.12/landscape/client/package/changer.py
1061-@@ -173,7 +173,7 @@ class PackageChanger(PackageTaskHandler)
1062- hash_ids = {}
1063- for hash, id, deb in binaries:
1064- create_binary_file(os.path.join(binaries_path, "%d.deb" % id),
1065-- base64.decodestring(deb))
1066-+ base64.decodebytes(deb))
1067- hash_ids[hash] = id
1068- self._store.set_hash_ids(hash_ids)
1069- self._facade.add_channel_deb_dir(binaries_path)
1070---- landscape-client-19.12.orig/landscape/client/package/tests/test_changer.py
1071-+++ landscape-client-19.12/landscape/client/package/tests/test_changer.py
1072-@@ -849,9 +849,9 @@ class AptPackageChangerTest(LandscapeTes
1073-
1074- binaries_path = self.config.binaries_path
1075- self.assertFileContent(os.path.join(binaries_path, "111.deb"),
1076-- base64.decodestring(PKGDEB1))
1077-+ base64.decodebytes(PKGDEB1))
1078- self.assertFileContent(os.path.join(binaries_path, "222.deb"),
1079-- base64.decodestring(PKGDEB2))
1080-+ base64.decodebytes(PKGDEB2))
1081- self.assertEqual(
1082- self.facade.get_channels(),
1083- self.get_binaries_channels(binaries_path))
1084---- landscape-client-19.12.orig/landscape/lib/apt/package/testing.py
1085-+++ landscape-client-19.12/landscape/lib/apt/package/testing.py
1086-@@ -322,9 +322,9 @@ PKGDEB_OR_RELATIONS = (
1087- ).encode("ascii")
1088-
1089-
1090--HASH1 = base64.decodestring(b"/ezv4AefpJJ8DuYFSq4RiEHJYP4=")
1091--HASH2 = base64.decodestring(b"glP4DwWOfMULm0AkRXYsH/exehc=")
1092--HASH3 = base64.decodestring(b"NJM05mj86veaSInYxxqL1wahods=")
1093-+HASH1 = base64.decodebytes(b"/ezv4AefpJJ8DuYFSq4RiEHJYP4=")
1094-+HASH2 = base64.decodebytes(b"glP4DwWOfMULm0AkRXYsH/exehc=")
1095-+HASH3 = base64.decodebytes(b"NJM05mj86veaSInYxxqL1wahods=")
1096- HASH_MINIMAL = b"6\xce\x8f\x1bM\x82MWZ\x1a\xffjAc(\xdb(\xa1\x0eG"
1097- HASH_SIMPLE_RELATIONS = (
1098- b"'#\xab&k\xe6\xf5E\xcfB\x9b\xceO7\xe6\xec\xa9\xddY\xaa")
1099-@@ -339,7 +339,7 @@ HASH_OR_RELATIONS = (
1100- def create_deb(target_dir, pkg_name, pkg_data):
1101- """Create a Debian package in the specified C{target_dir}."""
1102- path = os.path.join(target_dir, pkg_name)
1103-- data = base64.decodestring(pkg_data)
1104-+ data = base64.decodebytes(pkg_data)
1105- create_binary_file(path, data)
1106-
1107-
1108diff --git a/debian/patches/replace-tostring.patch b/debian/patches/replace-tostring.patch
1109deleted file mode 100644
1110index 834eb53..0000000
1111--- a/debian/patches/replace-tostring.patch
1112+++ /dev/null
1113@@ -1,25 +0,0 @@
1114-Index: landscape-client-19.12/landscape/lib/network.py
1115-===================================================================
1116---- landscape-client-19.12.orig/landscape/lib/network.py
1117-+++ landscape-client-19.12/landscape/lib/network.py
1118-@@ -11,7 +11,7 @@ import errno
1119- import logging
1120-
1121- import netifaces
1122--from twisted.python.compat import long
1123-+from twisted.python.compat import long, _PY3
1124-
1125- __all__ = ["get_active_device_info", "get_network_traffic"]
1126-
1127-@@ -246,7 +246,10 @@ def get_network_interface_speed(sock, in
1128- speed = -1
1129- try:
1130- fcntl.ioctl(sock, SIOCETHTOOL, packed) # Status ioctl() call
1131-- res = status_cmd.tostring()
1132-+ if _PY3:
1133-+ res = status_cmd.tobytes()
1134-+ else:
1135-+ res = status_cmd.tostring()
1136- speed, duplex = struct.unpack("12xHB28x", res)
1137- except (IOError, OSError) as e:
1138- if e.errno == errno.EPERM:
1139diff --git a/debian/patches/series b/debian/patches/series
1140index dcad3a3..e69de29 100644
1141--- a/debian/patches/series
1142+++ b/debian/patches/series
1143@@ -1,7 +0,0 @@
1144-0001-Handle-EINVAL-error-of-SIOCETHTOOL-ioctl.patch
1145-0002-lp1870087-stale-locks.patch
1146-py3.9.patch
1147-0003-clean-publisher-shutdown.patch
1148-replace-tostring.patch
1149-1962539_twisted_py3.patch
1150-lp1903776-release-upgrade.patch
1151diff --git a/debian/rules b/debian/rules
1152index 37cce8e..ad30f5c 100755
1153--- a/debian/rules
1154+++ b/debian/rules
1155@@ -23,4 +23,4 @@ override_dh_auto_install:
1156 install -D -o root -g root -m 755 apt-update/apt-update $(CURDIR)/$(root_dir)$(LIBDIR)/apt-update
1157
1158 override_dh_installsystemd:
1159- dh_installsystemd --no-enable --no-start
1160+ dh_installsystemd --no-enable --no-start --no-stop-on-upgrade
1161diff --git a/example.conf b/example.conf
1162index 3965d29..4b7fd94 100644
1163--- a/example.conf
1164+++ b/example.conf
1165@@ -174,3 +174,11 @@ manager_plugins = ALL
1166 #
1167 # By default, all usernames are allowed.
1168 script_users = ALL
1169+
1170+
1171+# The maximum script output length transmitted to landscape
1172+# Output over this limit is truncated
1173+#
1174+# The default is 512kB
1175+# 2MB is allowed in this example
1176+#script_output_limit=2048
1177diff --git a/landscape/__init__.py b/landscape/__init__.py
1178index d76ad6c..23f8f79 100644
1179--- a/landscape/__init__.py
1180+++ b/landscape/__init__.py
1181@@ -1,5 +1,5 @@
1182 DEBIAN_REVISION = ""
1183-UPSTREAM_VERSION = "18.01"
1184+UPSTREAM_VERSION = "23.02"
1185 VERSION = "%s%s" % (UPSTREAM_VERSION, DEBIAN_REVISION)
1186
1187 # The minimum server API version that all Landscape servers are known to speak
1188diff --git a/landscape/client/broker/exchange.py b/landscape/client/broker/exchange.py
1189index ee6d0e9..924cb9a 100644
1190--- a/landscape/client/broker/exchange.py
1191+++ b/landscape/client/broker/exchange.py
1192@@ -347,11 +347,12 @@ import logging
1193 from landscape.lib.hashlib import md5
1194
1195 from twisted.internet.defer import Deferred, succeed
1196-from twisted.python.compat import _PY3
1197+from landscape.lib.compat import _PY3
1198
1199+from landscape.lib.backoff import ExponentialBackoff
1200 from landscape.lib.fetch import HTTPCodeError, PyCurlError
1201 from landscape.lib.format import format_delta
1202-from landscape.lib.message import got_next_expected, ANCIENT
1203+from landscape.lib.message import got_next_expected, RESYNC
1204 from landscape.lib.versioning import is_version_higher, sort_versions
1205
1206 from landscape import DEFAULT_SERVER_API, SERVER_API, CLIENT_API
1207@@ -397,6 +398,7 @@ class MessageExchange(object):
1208 self._exchange_interval = config.exchange_interval
1209 self._urgent_exchange_interval = config.urgent_exchange_interval
1210 self._max_messages = max_messages
1211+ self._max_log_text_bytes = 100000 # 100KB
1212 self._notification_id = None
1213 self._exchange_id = None
1214 self._exchanging = False
1215@@ -406,6 +408,7 @@ class MessageExchange(object):
1216 self._message_handlers = {}
1217 self._exchange_store = exchange_store
1218 self._stopped = False
1219+ self._backoff_counter = ExponentialBackoff(300, 7200) # 5 to 120 min
1220
1221 self.register_message("accepted-types", self._handle_accepted_types)
1222 self.register_message("resynchronize", self._handle_resynchronize)
1223@@ -436,6 +439,20 @@ class MessageExchange(object):
1224
1225 return result
1226
1227+ def truncate_message_field(self, field, message):
1228+ """
1229+ Truncates message field value based on length
1230+ """
1231+ if field in message:
1232+ value = message[field]
1233+ if isinstance(value, str):
1234+ # Note this is an approximation based on 1 byte chars
1235+ max_bytes = self._max_log_text_bytes
1236+ if len(value) > max_bytes:
1237+ value = value[:max_bytes]
1238+ value += '...MESSAGE TRUNCATED DUE TO SIZE'
1239+ message[field] = value
1240+
1241 def send(self, message, urgent=False):
1242 """Include a message to be sent in an exchange.
1243
1244@@ -451,6 +468,10 @@ class MessageExchange(object):
1245 % message.get('operation-id'))
1246 return None
1247
1248+ # These fields sometimes have really long output we need to trim
1249+ self.truncate_message_field('err', message)
1250+ self.truncate_message_field('result-text', message)
1251+
1252 if "timestamp" not in message:
1253 message["timestamp"] = int(self._reactor.time())
1254 message_id = self._message_store.add(message)
1255@@ -567,6 +588,7 @@ class MessageExchange(object):
1256 self._urgent_exchange = False
1257 self._handle_result(payload, result)
1258 self._message_store.record_success(int(self._reactor.time()))
1259+ self._backoff_counter.decrease()
1260 else:
1261 self._reactor.fire("exchange-failed")
1262 logging.info("Message exchange failed.")
1263@@ -586,6 +608,20 @@ class MessageExchange(object):
1264 self.exchange()
1265 return
1266
1267+ if isinstance(error, HTTPCodeError):
1268+ if error.http_code == 429 or (500 <= error.http_code <= 599):
1269+ # We add an exponentially increasing delay ("backoff") if
1270+ # the server is overloaded to decrease load. We assume that
1271+ # any server backend error is deserving backoff, including
1272+ # 429 which is sent from the server on purpose to trigger
1273+ # the backoff. Whether the server is down or overloaded
1274+ # (503), has a server bug crashing the response (500),
1275+ # backing off should have no ill effect and help the
1276+ # service to recover. Client-configuration related errors
1277+ # (501, 505) are probably fine to throttle too as a higher
1278+ # rate won't help resolve them
1279+ self._backoff_counter.increase()
1280+
1281 ssl_error = False
1282 if isinstance(error, PyCurlError) and error.error_code == 60:
1283 # The error returned is an SSL error, most likely the server
1284@@ -642,6 +678,11 @@ class MessageExchange(object):
1285 interval = self._config.urgent_exchange_interval
1286 else:
1287 interval = self._config.exchange_interval
1288+ backoff_delay = self._backoff_counter.get_random_delay()
1289+ if backoff_delay:
1290+ logging.warning("Server is busy. Backing off client for {} "
1291+ "seconds".format(backoff_delay))
1292+ interval += backoff_delay
1293
1294 if self._notification_id is not None:
1295 self._reactor.cancel_call(self._notification_id)
1296@@ -746,9 +787,9 @@ class MessageExchange(object):
1297 next_expected += len(payload["messages"])
1298
1299 message_store_state = got_next_expected(message_store, next_expected)
1300- if message_store_state == ANCIENT:
1301- # The server has probably lost some data we sent it. The
1302- # slate has been wiped clean (by got_next_expected), now
1303+ if message_store_state == RESYNC:
1304+ # The server has probably lost some data we sent it. Or next
1305+ # expected is too high so the sequences are out of sync. Now
1306 # let's fire an event to tell all the plugins that they
1307 # ought to generate new messages so the server gets some
1308 # up-to-date data.
1309@@ -800,8 +841,7 @@ class MessageExchange(object):
1310 # actually expect it to be a string. Some unit tests set it to
1311 # a regular string (since there is no difference between strings
1312 # and bytes in Python 2), so we check the type before converting.
1313- if _PY3 and isinstance(message["type"], bytes):
1314- message["type"] = message["type"].decode("ascii")
1315+ message["type"] = maybe_bytes(message["type"])
1316 self.handle_message(message)
1317 sequence += 1
1318 message_store.set_server_sequence(sequence)
1319@@ -843,8 +883,9 @@ class MessageExchange(object):
1320
1321 self._reactor.fire("message", message)
1322 # This has plan interference! but whatever.
1323- if message["type"] in self._message_handlers:
1324- for handler in self._message_handlers[message["type"]]:
1325+ message_type = maybe_bytes(message["type"])
1326+ if message_type in self._message_handlers:
1327+ for handler in self._message_handlers[message_type]:
1328 handler(message)
1329
1330 def register_client_accepted_message_type(self, type):
1331@@ -866,3 +907,10 @@ def get_accepted_types_diff(old_types, new_types):
1332 diff.extend(["%s" % type for type in stable_types])
1333 diff.extend(["-%s" % type for type in removed_types])
1334 return " ".join(diff)
1335+
1336+
1337+def maybe_bytes(thing):
1338+ """Return a py3 ascii string from maybe py2 bytes."""
1339+ if _PY3 and isinstance(thing, bytes):
1340+ return thing.decode("ascii")
1341+ return thing
1342diff --git a/landscape/client/broker/registration.py b/landscape/client/broker/registration.py
1343index 2caef90..864ce99 100644
1344--- a/landscape/client/broker/registration.py
1345+++ b/landscape/client/broker/registration.py
1346@@ -8,10 +8,13 @@ the machinery in this module will notice that we have no identification
1347 credentials yet and that the server accepts registration messages, so it
1348 will craft an appropriate one and send it out.
1349 """
1350+import json
1351 import logging
1352
1353 from twisted.internet.defer import Deferred
1354
1355+from landscape.client.broker.exchange import maybe_bytes
1356+from landscape.client.monitor.ubuntuproinfo import get_ubuntu_pro_info
1357 from landscape.lib.juju import get_juju_info
1358 from landscape.lib.tag import is_valid_tag_list
1359 from landscape.lib.network import get_fqdn
1360@@ -106,6 +109,7 @@ class RegistrationHandler(object):
1361 self._should_register = None
1362 self._fetch_async = fetch_async
1363 self._juju_data = None
1364+ self._clone_secure_id = None
1365
1366 def should_register(self):
1367 id = self._identity
1368@@ -191,6 +195,13 @@ class RegistrationHandler(object):
1369 "container-info": get_container_info(),
1370 "vm-info": get_vm_info()}
1371
1372+ if self._clone_secure_id:
1373+ # We use the secure id here because the registration is encrypted
1374+ # and the insecure id has been already exposed to unencrypted
1375+ # http from the ping server. In addition it's more straightforward
1376+ # to get the computer from the server through it than the insecure
1377+ message["clone_secure_id"] = self._clone_secure_id
1378+
1379 if group:
1380 message["access_group"] = group
1381
1382@@ -212,6 +223,8 @@ class RegistrationHandler(object):
1383 with_tags = "and tags %s " % tags if tags else ""
1384 with_group = "in access group '%s' " % group if group else ""
1385
1386+ message["ubuntu_pro_info"] = json.dumps(get_ubuntu_pro_info())
1387+
1388 logging.info(
1389 u"Queueing message to register with account %r %s%s"
1390 "%s a password." % (
1391@@ -239,8 +252,9 @@ class RegistrationHandler(object):
1392 self._reactor.fire("resynchronize-clients")
1393
1394 def _handle_registration(self, message):
1395- if message["info"] in ("unknown-account", "max-pending-computers"):
1396- self._reactor.fire("registration-failed", reason=message["info"])
1397+ message_info = maybe_bytes(message["info"])
1398+ if message_info in ("unknown-account", "max-pending-computers"):
1399+ self._reactor.fire("registration-failed", reason=message_info)
1400
1401 def _handle_unknown_id(self, message):
1402 id = self._identity
1403@@ -248,18 +262,9 @@ class RegistrationHandler(object):
1404 if clone is None:
1405 logging.info("Client has unknown secure-id for account %s."
1406 % id.account_name)
1407- else:
1408+ else: # Save the secure id as the clone, and clear it so it's renewed
1409 logging.info("Client is clone of computer %s" % clone)
1410- # Set a new computer title so when a registration request will be
1411- # made, the pending computer UI will indicate that this is a clone
1412- # of another computer. There's no need to persist the changes since
1413- # a new registration will be requested immediately.
1414- if clone == self._config.computer_title:
1415- title = "%s (clone)" % self._config.computer_title
1416- else:
1417- title = "%s (clone of %s)" % (self._config.computer_title,
1418- clone)
1419- self._config.computer_title = title
1420+ self._clone_secure_id = id.secure_id
1421 id.secure_id = None
1422 id.insecure_id = None
1423
1424diff --git a/landscape/client/broker/server.py b/landscape/client/broker/server.py
1425index 2805ea5..1f8d708 100644
1426--- a/landscape/client/broker/server.py
1427+++ b/landscape/client/broker/server.py
1428@@ -46,6 +46,7 @@ Diagram::
1429 import logging
1430
1431 from twisted.internet.defer import Deferred
1432+from landscape.lib.compat import _PY3
1433
1434 from landscape.lib.twisted_util import gather_results
1435 from landscape.client.amp import remote
1436@@ -191,6 +192,14 @@ class BrokerServer(object):
1437 during the next regularly scheduled exchange.
1438 @return: The message identifier created when queuing C{message}.
1439 """
1440+ if b"type" in message and _PY3:
1441+ # XXX We are getting called by a python2 process.
1442+ # This occurs in the the specific case of a landscape-driven
1443+ # release upgrade to bionic, where the upgrading process is still
1444+ # running under python2.7. Therefore we do backward-compatible
1445+ # translation of byte keys in the sent message
1446+ message = {k.decode("ascii"): v for k, v in message.items()}
1447+ message["type"] = message["type"].decode("ascii")
1448 if isinstance(session_id, bool) and message["type"] in (
1449 "operation-result", "change-packages-result"):
1450 # XXX This means we're performing a Landscape-driven upgrade and
1451diff --git a/landscape/client/broker/service.py b/landscape/client/broker/service.py
1452index 6e89414..7e36149 100644
1453--- a/landscape/client/broker/service.py
1454+++ b/landscape/client/broker/service.py
1455@@ -82,10 +82,11 @@ class BrokerService(LandscapeService):
1456
1457 def stopService(self):
1458 """Stop the broker."""
1459- self.publisher.stop()
1460+ deferred = self.publisher.stop()
1461 self.exchanger.stop()
1462 self.pinger.stop()
1463 super(BrokerService, self).stopService()
1464+ return deferred
1465
1466
1467 def run(args):
1468diff --git a/landscape/client/broker/tests/badprivate.ssl b/landscape/client/broker/tests/badprivate.ssl
1469index db67ad4..d5e868c 100644
1470--- a/landscape/client/broker/tests/badprivate.ssl
1471+++ b/landscape/client/broker/tests/badprivate.ssl
1472@@ -1,15 +1,27 @@
1473 -----BEGIN RSA PRIVATE KEY-----
1474-MIICXgIBAAKBgQDGYFWP2Ine2OFIPjX+Tu+S403KW63EWq/I1DYXiezLoUpYPed3
1475-0tAkAXH1gOwQZbARFlUn0LgvXDSpuQLvgKQZwP/e1D8SvZUZ6nexW+aYlPE9kjd1
1476-dhK1xpe1h5y09AjCz02xxzcFzrJrJ47uU7vV+FGArE8FFh3hO+dz0/PmZQIDAQAB
1477-AoGBAKfv+983yJfgcO9QwzLULlrilQNfk36r6y6QAG7y84T7uU10spSs4kno80mL
1478-58yF2YTNrC91scdePrMEDikldUVcCqtPYcZKHyw5+4aGaDDO244tznexOQnQcNIe
1479-2BbLFuh+jmJpoFIY/H7EsLQQzn6+6dGPnYGBQfiyitWfAXRNAkEA/ShQkYCRAHgq
1480-g6WBIYsw/ISQydhiMiKrL2ZUXERT+pWU9MoSdMskgyMi3S7wzwJQXkrHA36q8QkL
1481-+H8n5K+f5wJBAMiajfEtv0wRW0awX40qJtuqW3cSKeGHBH9mMObcRJd5OcK6giC/
1482-Cc5st/ZcuE/8i4r44DfeC+cwY6QdIqI8rdMCQQCKuq78LWJIyZEyt12+ThK4LsVR
1483-d1zIcKsyvHb6YQ9MQPBx/NKEYlZN7tFKOFEKgBAevAe3aJCwqe5/bN8luQB9AkEA
1484-uQVD8bR+AgzoIPS/zJWaLXSc09/e3PIJBfAdHnD+mq7mxWH8b3OD+e5wZjvyi2Ok
1485-2NLfCug0FlGdNVrh/Lz2nQJATdcNvHNzJcWOHe05lo+xAqkjz73FWGpPNrdXRigG
1486-YnjIsZVy4k48xIxPhT2rC44yo1iPEP5EnHCE2bLyUlTAYA==
1487+MIIEowIBAAKCAQEA1f55iafJwi9EEL1jp8lX89SFVsordyhAJzQP+rRElFIcLSXU
1488+2qC4whUErO3h3nKJXkIx4bHaQSn+/UKawr/kSVJI2QT3vimI/JTD5srxZJUrWEgc
1489+slQNihx27jsRfpVeFK2BV2UpBYg65D6UhWyxNmoenBszBRRi11QnOF2VVX0BQLjO
1490+lOP5mysSnq+VGEIEhViVICBKVb3dYle/LTh9uBH4oBBl4RvmzRZ/WUXqVLao8ryr
1491+hciYDmCEpJH6tE4b/aEQNxou7nZqPJ91fkKZh4SmlhzXMFsjpQALZNYAxYZY6Kr4
1492+9Z+Xw8ckf3qqYfi8KovjCec7gewhCdZNpBxxOQIDAQABAoIBAHvT5DpOmEZAmY9i
1493+OB9oN/fFO18sX5h09yJ4UuLMm36EQP+zC4dzR1YvWWRDxta0yl57yWeDRfs9NOsS
1494+NoGJDq2K6tKBuGYWnMkjwHR1bNe6JbnRCKH8V1VbAUr7bTUlc6pdeCG9TM6BtSpM
1495+OB849Ra6s3m7l3tR/5wAey13oak0PHM19c2Iv3Sniydt9toWyvuVbXWzOuupKMah
1496++eFkD0wpEgigL7gS4ToYXMqWn2LNHYsUSXKyuhtR0B/Ow6fwwb2KYxG5a5/hFI+E
1497+URxFS148R/BtywUmxbzycTvZbx8XY7/lZjAlwYRCsYEF2EmmRbRGrHWOIuoHBMJu
1498+49OMFu0CgYEA9P/EgmPeuIhrk3WFKnUTvHHTiW/Knv5xWRpkV+pXyGh4l+1F9lrj
1499+WxORh+ws/6IpZevVVkUQxsd/Q4Uda7LlocS/YNSFC+LV471Y854+Wonbun8JBd+9
1500+GAFtG6o2XOaezp8zf8oDVqnfIx2qpySll3aNY7JrNZngtaE2bKz3sRMCgYEA35pO
1501+47eTLxgIoFwr/M5cCT1nZ2FZUTBET9grrBgE5KTjtNbFAYJh7mP1lhILI58q7Lan
1502+k9m5YoqAGsoKrilA715ks6vBQYWhlsg6p+rCvBTIYp4kxZwhTad5fFqXyhokwUZY
1503+0ZJMtxYxK0xIZIEkuyzdzLiuYrKupFAB+3p+6gMCgYARQoeMjA6fv3ScsdXM1Oys
1504+BPTbJNYId3JyzYouK2M9yiZcxal9HpAP1YQWKExPQhRais+/wSPabSmJDzKwaK0G
1505+xX6aCr7IxJU+8xL2LrrD1Bx3ugVftZBzxX3zSf2Ec/bSJaMSKKAtldATgD6Kgels
1506+jzyMvoARCaMsCIx2AYV9owKBgQCdd6UA9wHfE3TXwbF0mrr0Ats0Ubk91Nj2xcyT
1507+qGKhxoFZlDovAuwGnzyPT+uqTWhERamkFJtaiyEGPKzi08iYCgivA1DY3MvcTOwJ
1508+3uj+3T/1O1u4EmjdsAh9C6uDt3+U4P6hr/74nNdJn7IHnW8JpeIZTyH3/c/BhVqw
1509+CCcikwKBgD7Kbl1AaK+6/+oDYunLOzxfDXkxkiXRDeBpvgOIW1G1F+bNJyO800oW
1510+eZDtGU01Ze8gjNUbbYX39q4rCixtBHuJEwAH8J3grvtaHYpwjfuYgnhaXM1MHJo2
1511+kxfrQTDg/v3NjaccxCESeV1jjxz5hS2GrvqOvZrwyVtgi/r71cLJ
1512 -----END RSA PRIVATE KEY-----
1513diff --git a/landscape/client/broker/tests/badpublic.ssl b/landscape/client/broker/tests/badpublic.ssl
1514index 0a9b87d..9451fcd 100644
1515--- a/landscape/client/broker/tests/badpublic.ssl
1516+++ b/landscape/client/broker/tests/badpublic.ssl
1517@@ -1,5 +1,5 @@
1518 -----BEGIN CERTIFICATE-----
1519-MIIDzjCCAzegAwIBAgIJANqT3vXxSVFjMA0GCSqGSIb3DQEBBQUAMIGhMQswCQYD
1520+MIIE0zCCA7ugAwIBAgIJANqT3vXxSVFjMA0GCSqGSIb3DQEBCwUAMIGhMQswCQYD
1521 VQQGEwJCUjEPMA0GA1UECBMGUGFyYW5hMREwDwYDVQQHEwhDdXJpdGliYTEhMB8G
1522 A1UEChMYRmFrZSBMYW5kc2NhcGUgKFRlc3RpbmcpMREwDwYDVQQLEwhTZWN1cml0
1523 eTESMBAGA1UEAxMJbG9jYWxob3N0MSQwIgYJKoZIhvcNAQkBFhVhbmRyZWFzQGNh
1524@@ -7,17 +7,22 @@ bm9uaWNhbC5jb20wHhcNMDkwMTA5MTUyNTAwWhcNMTkwMTA3MTUyNTAwWjCBoTEL
1525 MAkGA1UEBhMCQlIxDzANBgNVBAgTBlBhcmFuYTERMA8GA1UEBxMIQ3VyaXRpYmEx
1526 ITAfBgNVBAoTGEZha2UgTGFuZHNjYXBlIChUZXN0aW5nKTERMA8GA1UECxMIU2Vj
1527 dXJpdHkxEjAQBgNVBAMTCWxvY2FsaG9zdDEkMCIGCSqGSIb3DQEJARYVYW5kcmVh
1528-c0BjYW5vbmljYWwuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDGYFWP
1529-2Ine2OFIPjX+Tu+S403KW63EWq/I1DYXiezLoUpYPed30tAkAXH1gOwQZbARFlUn
1530-0LgvXDSpuQLvgKQZwP/e1D8SvZUZ6nexW+aYlPE9kjd1dhK1xpe1h5y09AjCz02x
1531-xzcFzrJrJ47uU7vV+FGArE8FFh3hO+dz0/PmZQIDAQABo4IBCjCCAQYwHQYDVR0O
1532-BBYEFF4A8+YHCLAt19OtWTjIjBKzLUokMIHWBgNVHSMEgc4wgcuAFF4A8+YHCLAt
1533-19OtWTjIjBKzLUokoYGnpIGkMIGhMQswCQYDVQQGEwJCUjEPMA0GA1UECBMGUGFy
1534-YW5hMREwDwYDVQQHEwhDdXJpdGliYTEhMB8GA1UEChMYRmFrZSBMYW5kc2NhcGUg
1535-KFRlc3RpbmcpMREwDwYDVQQLEwhTZWN1cml0eTESMBAGA1UEAxMJbG9jYWxob3N0
1536-MSQwIgYJKoZIhvcNAQkBFhVhbmRyZWFzQGNhbm9uaWNhbC5jb22CCQDak9718UlR
1537-YzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4GBABszkA3CCzt+nTOX+A7/
1538-I98DvI0W1Ss0J+Tq+diLr+kw6Z5ZTj5hrIS/x6XhVHjpim4724UBXA0Sels4JXbw
1539-hhJovuncExce316gAol/9eEzTffZ9mt1jZQy9LL7IAENiobnsj2F65zNaJzXp5UC
1540-rE/h/xIxz9rAmXtVOWHqZLcw
1541+c0BjYW5vbmljYWwuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
1542+1f55iafJwi9EEL1jp8lX89SFVsordyhAJzQP+rRElFIcLSXU2qC4whUErO3h3nKJ
1543+XkIx4bHaQSn+/UKawr/kSVJI2QT3vimI/JTD5srxZJUrWEgcslQNihx27jsRfpVe
1544+FK2BV2UpBYg65D6UhWyxNmoenBszBRRi11QnOF2VVX0BQLjOlOP5mysSnq+VGEIE
1545+hViVICBKVb3dYle/LTh9uBH4oBBl4RvmzRZ/WUXqVLao8ryrhciYDmCEpJH6tE4b
1546+/aEQNxou7nZqPJ91fkKZh4SmlhzXMFsjpQALZNYAxYZY6Kr49Z+Xw8ckf3qqYfi8
1547+KovjCec7gewhCdZNpBxxOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYEFF4A8+YHCLAt
1548+19OtWTjIjBKzLUokMIHWBgNVHSMEgc4wgcuAFF4A8+YHCLAt19OtWTjIjBKzLUok
1549+oYGnpIGkMIGhMQswCQYDVQQGEwJCUjEPMA0GA1UECBMGUGFyYW5hMREwDwYDVQQH
1550+EwhDdXJpdGliYTEhMB8GA1UEChMYRmFrZSBMYW5kc2NhcGUgKFRlc3RpbmcpMREw
1551+DwYDVQQLEwhTZWN1cml0eTESMBAGA1UEAxMJbG9jYWxob3N0MSQwIgYJKoZIhvcN
1552+AQkBFhVhbmRyZWFzQGNhbm9uaWNhbC5jb22CCQDak9718UlRYzAMBgNVHRMEBTAD
1553+AQH/MA0GCSqGSIb3DQEBCwUAA4IBAQBGwsH3kzRp8hQMlhNBFjzGCfycS8VT/AIr
1554+VIORXzitEEovBqnPM5VT48sx7y7L7e9bmtBU9jv2aqk/PtRgTIl7tFcI5XeVzlsF
1555+HB3A5Y7ytJqp80bIZvKt8gEq2qaBmyIK9VIHBO56Yxb7lQhkWWZmVBeCBbOut6HI
1556+oBC8IX1hck/K3DjQh8gZnuJz7Ut1Pvb7uIGkfHWb6fGuYDcpZXj3uhRAKzRIZgdE
1557+xs28Om1dKO6CDeTVkT2OSPfj21EJ1hNXrPQfXlInbyjYgR72gvqN+kbj9sc1dUr5
1558+l0McTP+hrRFpRN/rUrqtOWtHFW+INXj/HAuK+i4MlwkQXgPOTmoy
1559 -----END CERTIFICATE-----
1560diff --git a/landscape/client/broker/tests/helpers.py b/landscape/client/broker/tests/helpers.py
1561index fc01020..367a03f 100644
1562--- a/landscape/client/broker/tests/helpers.py
1563+++ b/landscape/client/broker/tests/helpers.py
1564@@ -39,14 +39,16 @@ class BrokerConfigurationHelper(object):
1565 log_dir = test_case.makeDir()
1566 test_case.config_filename = os.path.join(test_case.makeDir(),
1567 "client.conf")
1568- open(test_case.config_filename, "w").write(
1569- "[client]\n"
1570- "url = http://localhost:91919\n"
1571- "computer_title = Some Computer\n"
1572- "account_name = some_account\n"
1573- "ping_url = http://localhost:91910\n"
1574- "data_path = %s\n"
1575- "log_dir = %s\n" % (data_path, log_dir))
1576+
1577+ with open(test_case.config_filename, "w") as fh:
1578+ fh.write(
1579+ "[client]\n"
1580+ "url = http://localhost:91919\n"
1581+ "computer_title = Some Computer\n"
1582+ "account_name = some_account\n"
1583+ "ping_url = http://localhost:91910\n"
1584+ "data_path = %s\n"
1585+ "log_dir = %s\n" % (data_path, log_dir))
1586
1587 bootstrap_list.bootstrap(data_path=data_path, log_dir=log_dir)
1588
1589diff --git a/landscape/client/broker/tests/private.ssl b/landscape/client/broker/tests/private.ssl
1590index 1484432..c103d59 100644
1591--- a/landscape/client/broker/tests/private.ssl
1592+++ b/landscape/client/broker/tests/private.ssl
1593@@ -1,15 +1,27 @@
1594 -----BEGIN RSA PRIVATE KEY-----
1595-MIICWwIBAAKBgQDX2VNEDZHtl5nimNocshar8pBmjqiGn9olCR2LcKifuJY4bFTg
1596-qib+Rr3v2DwDTbOMaquRSxFgwLJLCug3WclsGrYSPIsFCx+k3XhqM61JXEwrKuIp
1597-Js893XHkeg3SEFua/oVfDxNfJttoHW3FbsnDx5964kYwGExjJcH73GInUQIDAQAB
1598-AoGASiM9NEys6Lx/gJMbp2uL2fdwnak2PTc+iCX/XduOL34JKswawyfuSLwnlO/i
1599-fQf9OaeR0k/EYkUNeDUA2bIfOj6wWS8tamnX4fxL7A20y5VyqMMah8mcerZgtPdS
1600-7ZtYCbeijWSKpHgjALc2Hym7R68WZI+IHe0DQkcW6WxOMFkCQQD2jqHZn/Qtd62u
1601-mWVwIx6G7+Po5vzd86KyWWftdUtVCY9DmiX1rmWXbJhLnmaKCLkmHxyBvw7Biarr
1602-ZnCAafebAkEA4B2dSpLi7bAzjCa7JBtzV9kr1FVZOl2vA+9BqTAjCQu0b9VDEm8V
1603-x0061Z8rN7Og3ECGtKH/r3/4RnHUPpwJgwJAdyZQkvHYt4xJc8IPolRmcUFGu4u9
1604-Eammq1fHgJqZcBvxjvLUe1jvIXFKW+jNltFGYGTSiuUAxYi4/49+uJ/9FwJAGBB1
1605-/DTrcvQxhMH/5C+iYfNqtmD3tMGscjK1jTIjAOyl0kBG9GrDHuRXBesSW+fIxP2U
1606-uT6P0std4EqGrLZaewJAHT0n/3tXnsPjj+BMlC4ZkRKgPJ4I7zTU1XSlLY5zbMoV
1607-NvtHLlq7ttiarsH95xyge69uV1/zJVj/IiS71YY9PQ==
1608+MIIEogIBAAKCAQEAw1Cf8FvbgP/FBGyCPXrTw23qa1KqmDkEu3q/yqmVpBlHh93K
1609+UZu0T0WT7UJykEwOJqY+5G3Xaw53gWx7Lvz5nsI+X8Womd9NSqUkCiVq/TxeuWVm
1610+07+pohQi8OJMmdfflx2eDX+3xciHdB4+A7baefyGut1vGr1wTYA3pu3/agqNIcAo
1611+2F7LE13vnxgFzXCW0StdQeeT8KapOSJTZ8vxxkEKmXTixN7rpACP6GpD4jLTHZ/L
1612+YUxjfHJdRGBF6TH/4NaA5CnS2yJpGidBmeK8Jid/S54P7HXrcJTrC0xwdYFfUTEI
1613+25mBSqbEfGNqKhkYeJT/Pvy5TGdqAVSr4OjpEwIDAQABAoIBABw/4hI6xwHefJmK
1614+NEA+Lrjagghp2YDQ5m1TcMAYTSuB+IWfP68UDT1V+/JaJQXX6kgOzZPuizTRz9kp
1615+XpvKPTSINctWZG91C9HbFt5c0R+1hqHcF8ZSt29Y6EDdCmVKAu3xe7XKHkN+IJFb
1616++m5BGVKBgt8uPe6pLcAX5nS/gazNfyQ9s9zypMkCVSFoCOI+KkzqBCYPR/dqJUxP
1617+hnxDt78ndUJL9QIDIGu7aCr9UwQCwSiRTB3+fDt1iPk1RWl+zmqvHUscPdyY1oMe
1618+dvPNbF7Ea1nGjxzVG6+vyRHC1dQYSaqU8Ri26p0MBXGHCl8mq8tqV5ra30Q87ynV
1619+wNu2TvkCgYEA6UCUBoetPoT3ncp/QXOs2obPQJ9W7VRMIFpkiZNGs8Zivp06CY3e
1620+z7FqimDROKsFaYgDwPk5sOpzotnTA1/LsuoHK49Nux7Zn9RQak++6NrSozxvJ1P/
1621+ArpKkoTZNBbMPDghmib9gKUENjG1+6CkO5Z4tr0l9NyOBLz5j7oW2o8CgYEA1lzl
1622+X2olxIY2nxPlmCHwjXVD90FT9xmcAe4xtU96aXv8v+f+6V2rwSBr8oGI8oyNjZN9
1623+nLkT9fRVhGi1bf9VZYb8fWB8PtGumZRZcfqYDyMD/0xbT6vEwaNtPtqNf24t8jf3
1624+ijU+Yx2jIwE7cAtOKcUg0eMGRPzSF8NRgPTaWz0CgYBwXE1yP9VysnbdqfhXPTPd
1625+KOeZh6hGNz9crm6T30BFxaE3lWGpzI+ymRJrimv+0lOPHJhCU0w5Lxd5MVj23SSx
1626+EQ9XKncVVq0a0xnRvIyIezDQtYIN/eZwF/FoV1qSPxEvSRLWwUWIvPUkbhnuFtpG
1627+YhvQW5l3NO+s1KObWtc7fQKBgG4dIBJIU4hFLU/AB8ODQ69WmogrfbdD53iyY8Rw
1628+REBlWWs3ACHeZTj6r5jN44w8mQYtymu0QsWoMjmnE/OiIrrZgV/iLVCTo23u35eG
1629+E5BK+2WsUod1g8e4bIjJ+b+I2H9BMp5DRX3inod/vYmLtSYNxhMq3HCZsk5Uncxx
1630+eq09AoGAfAX7tLtZDxpvyzEEoNdtt3zf9OUYDb74vH0CLAwahDE6ZdK/BVoc+JJB
1631+Cm74/54z1nj7ubLNgMFuJbcgX/xthnuHcPSNH9FnPP//twYiuK1Zy67Q8aqFFxj4
1632+41H4idM4yLHJX1wjJDbufGHS/CjGg10Iy+gfFzqKqhkRhrdD3aI=
1633 -----END RSA PRIVATE KEY-----
1634diff --git a/landscape/client/broker/tests/public.ssl b/landscape/client/broker/tests/public.ssl
1635index 432b24d..ebd1682 100644
1636--- a/landscape/client/broker/tests/public.ssl
1637+++ b/landscape/client/broker/tests/public.ssl
1638@@ -1,22 +1,27 @@
1639 -----BEGIN CERTIFICATE-----
1640-MIIDnDCCAwWgAwIBAgIJAMjc7CvbvQHcMA0GCSqGSIb3DQEBCwUAMIGRMQswCQYD
1641+MIIEoTCCA4mgAwIBAgIJAMjc7CvbvQHcMA0GCSqGSIb3DQEBCwUAMIGRMQswCQYD
1642 VQQGEwJCUjEPMA0GA1UECBMGUGFyYW5hMREwDwYDVQQHEwhDdXJpdGliYTESMBAG
1643 A1UEChMJTGFuZHNjYXBlMRAwDgYDVQQLEwdUZXN0aW5nMRIwEAYDVQQDEwlsb2Nh
1644 bGhvc3QxJDAiBgkqhkiG9w0BCQEWFWFuZHJlYXNAY2Fub25pY2FsLmNvbTAeFw0x
1645 OTAxMzEyMjI4NTJaFw0yOTAxMjgyMjI4NTJaMIGRMQswCQYDVQQGEwJCUjEPMA0G
1646 A1UECBMGUGFyYW5hMREwDwYDVQQHEwhDdXJpdGliYTESMBAGA1UEChMJTGFuZHNj
1647 YXBlMRAwDgYDVQQLEwdUZXN0aW5nMRIwEAYDVQQDEwlsb2NhbGhvc3QxJDAiBgkq
1648-hkiG9w0BCQEWFWFuZHJlYXNAY2Fub25pY2FsLmNvbTCBnzANBgkqhkiG9w0BAQEF
1649-AAOBjQAwgYkCgYEA19lTRA2R7ZeZ4pjaHLIWq/KQZo6ohp/aJQkdi3Con7iWOGxU
1650-4Kom/ka979g8A02zjGqrkUsRYMCySwroN1nJbBq2EjyLBQsfpN14ajOtSVxMKyri
1651-KSbPPd1x5HoN0hBbmv6FXw8TXybbaB1txW7Jw8efeuJGMBhMYyXB+9xiJ1ECAwEA
1652-AaOB+TCB9jAdBgNVHQ4EFgQU3eUz2XxK1J/oavkn/hAvYfGOZM0wgcYGA1UdIwSB
1653-vjCBu4AU3eUz2XxK1J/oavkn/hAvYfGOZM2hgZekgZQwgZExCzAJBgNVBAYTAkJS
1654-MQ8wDQYDVQQIEwZQYXJhbmExETAPBgNVBAcTCEN1cml0aWJhMRIwEAYDVQQKEwlM
1655-YW5kc2NhcGUxEDAOBgNVBAsTB1Rlc3RpbmcxEjAQBgNVBAMTCWxvY2FsaG9zdDEk
1656-MCIGCSqGSIb3DQEJARYVYW5kcmVhc0BjYW5vbmljYWwuY29tggkAyNzsK9u9Adww
1657-DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOBgQCUq4lOtq19Q/NQmSdSQvpB
1658-GWj3NKkpsH6sDlzjfGTVDqbL6buSUq4fTmXfrx5ce/Y+APhfAINAwIL/zjCSV3zI
1659-EdhMG3BqMBmrQ60YfN7Z3drqfFzlg2yEVd/nJwrppjAI58KJgamN12WZ0eQTV8FR
1660-co8+OnqJEOPM8cMg7VEbtQ==
1661+hkiG9w0BCQEWFWFuZHJlYXNAY2Fub25pY2FsLmNvbTCCASIwDQYJKoZIhvcNAQEB
1662+BQADggEPADCCAQoCggEBAMNQn/Bb24D/xQRsgj1608Nt6mtSqpg5BLt6v8qplaQZ
1663+R4fdylGbtE9Fk+1CcpBMDiamPuRt12sOd4Fsey78+Z7CPl/FqJnfTUqlJAolav08
1664+XrllZtO/qaIUIvDiTJnX35cdng1/t8XIh3QePgO22nn8hrrdbxq9cE2AN6bt/2oK
1665+jSHAKNheyxNd758YBc1wltErXUHnk/CmqTkiU2fL8cZBCpl04sTe66QAj+hqQ+Iy
1666+0x2fy2FMY3xyXURgRekx/+DWgOQp0tsiaRonQZnivCYnf0ueD+x163CU6wtMcHWB
1667+X1ExCNuZgUqmxHxjaioZGHiU/z78uUxnagFUq+Do6RMCAwEAAaOB+TCB9jAdBgNV
1668+HQ4EFgQU3eUz2XxK1J/oavkn/hAvYfGOZM0wgcYGA1UdIwSBvjCBu4AU3eUz2XxK
1669+1J/oavkn/hAvYfGOZM2hgZekgZQwgZExCzAJBgNVBAYTAkJSMQ8wDQYDVQQIEwZQ
1670+YXJhbmExETAPBgNVBAcTCEN1cml0aWJhMRIwEAYDVQQKEwlMYW5kc2NhcGUxEDAO
1671+BgNVBAsTB1Rlc3RpbmcxEjAQBgNVBAMTCWxvY2FsaG9zdDEkMCIGCSqGSIb3DQEJ
1672+ARYVYW5kcmVhc0BjYW5vbmljYWwuY29tggkAyNzsK9u9AdwwDAYDVR0TBAUwAwEB
1673+/zANBgkqhkiG9w0BAQsFAAOCAQEAnHA/iAGqfjEVlB5PJbYkKNasaflJJnT34BUZ
1674+QdArozYCbC3dsysu+BkKEQ74FbDb/ZV0QhtV2/qYRlPsauWIiWU3gYk9wVjJdaBk
1675++MEViKOLV8CYe7ZA1OFNRQjfWUAiuRJdSsxnvOHcqcxrJT/hFmFz1OWyqcwfHBZY
1676+4+8A7Byuy7x0O3qOWT2gaXpFL9neqtla1nEa1egJTZLrMF8Pi3vIpFthEgQ+yXoW
1677+kBdWiYuMIry7YtAqBb4WgursCRftOCkPRYh3MVryqCDsVtOfGW2RO+YZnQgUpeTX
1678+a6kHrB1hh+p8Az9pPWQVPb4w/jn4L2kEjTNpUxYWVB8Sc8GtGA==
1679 -----END CERTIFICATE-----
1680diff --git a/landscape/client/broker/tests/test_exchange.py b/landscape/client/broker/tests/test_exchange.py
1681index 20ee434..3736bd4 100644
1682--- a/landscape/client/broker/tests/test_exchange.py
1683+++ b/landscape/client/broker/tests/test_exchange.py
1684@@ -145,6 +145,50 @@ class MessageExchangeTest(LandscapeTest):
1685 self.mstore.add_pending_offset(1)
1686 self.assertFalse(self.mstore.is_pending(message_id))
1687
1688+ def test_send_big_message_trimmed_err(self):
1689+ """
1690+ When package reporter sends error, message is trimmed if too long
1691+ """
1692+ self.mstore.set_accepted_types(["package-reporter-result"])
1693+ self.exchanger._max_log_text_bytes = 5
1694+ self.exchanger.send({"type": "package-reporter-result", "err": "E"*10,
1695+ "code": 0})
1696+ self.exchanger.exchange()
1697+ self.assertEqual(len(self.transport.payloads), 1)
1698+ messages = self.transport.payloads[0]["messages"]
1699+ self.assertIn('TRUNCATED', messages[0]['err'])
1700+ self.assertIn('EEEEE', messages[0]['err'])
1701+ self.assertNotIn('EEEEEE', messages[0]['err'])
1702+
1703+ def test_send_big_message_trimmed_result(self):
1704+ """
1705+ When an activity sends result log, message is trimmed if too long
1706+ """
1707+ self.mstore.set_accepted_types(["operation-result"])
1708+ self.exchanger._max_log_text_bytes = 5
1709+ self.exchanger.send({"type": "operation-result", "result-text": "E"*10,
1710+ "code": 0, "status": 0, "operation-id": 0})
1711+ self.exchanger.exchange()
1712+ self.assertEqual(len(self.transport.payloads), 1)
1713+ messages = self.transport.payloads[0]["messages"]
1714+ self.assertIn('TRUNCATED', messages[0]['result-text'])
1715+ self.assertIn('EEEEE', messages[0]['result-text'])
1716+ self.assertNotIn('EEEEEE', messages[0]['result-text'])
1717+
1718+ def test_send_small_message_not_trimmed(self):
1719+ """
1720+ If message is below length, nothing should happen
1721+ """
1722+ self.mstore.set_accepted_types(["package-reporter-result"])
1723+ self.exchanger._max_log_text_bytes = 4
1724+ self.exchanger.send({"type": "package-reporter-result", "err": "E"*4,
1725+ "code": 0})
1726+ self.exchanger.exchange()
1727+ self.assertEqual(len(self.transport.payloads), 1)
1728+ messages = self.transport.payloads[0]["messages"]
1729+ self.assertNotIn('TRUNCATED', messages[0]['err'])
1730+ self.assertIn('EEEE', messages[0]['err'])
1731+
1732 def test_wb_include_accepted_types(self):
1733 """
1734 Every payload from the client needs to specify an ID which
1735@@ -324,6 +368,88 @@ class MessageExchangeTest(LandscapeTest):
1736 self.assertEqual(payload["sequence"], 1)
1737 self.assertEqual(payload["next-expected-sequence"], 0)
1738
1739+ @mock.patch("landscape.client.broker.store.MessageStore"
1740+ ".delete_old_messages")
1741+ def test_pending_offset_when_next_expected_too_high(self,
1742+ mock_rm_all_messages):
1743+ '''
1744+ When next expected sequence received from server is too high, then the
1745+ pending offset should reset to zero. This will cause the client to
1746+ resend the pending messages.
1747+ '''
1748+
1749+ self.mstore.set_accepted_types(["data"])
1750+ self.mstore.add({"type": "data", "data": 0})
1751+ self.mstore.add({"type": "data", "data": 1})
1752+
1753+ self.exchanger.exchange()
1754+
1755+ self.assertEqual(self.mstore.get_pending_offset(), 2)
1756+
1757+ self.mstore.add({"type": "data", "data": 2})
1758+
1759+ self.transport.next_expected_sequence = 100
1760+
1761+ # Confirm pending offset is reset so that messages are sent again
1762+ self.exchanger.exchange()
1763+ self.assertEqual(self.mstore.get_pending_offset(), 0)
1764+
1765+ # This function flushes the queue, otherwise an offset of 0 will resend
1766+ # previous messages that were already successful
1767+ self.assertTrue(mock_rm_all_messages.called)
1768+
1769+ def test_payloads_when_next_expected_too_high(self):
1770+ '''
1771+ When next expected sequence received from server is too high, then the
1772+ current messages should get sent again since we don't have confirmation
1773+ that the server received it. Also previous messages should not get
1774+ repeated.
1775+ '''
1776+
1777+ self.mstore.set_accepted_types(["data"])
1778+
1779+ message0 = {"type": "data", "data": 0}
1780+ self.mstore.add(message0)
1781+ self.exchanger.exchange()
1782+
1783+ message1 = {"type": "data", "data": 1}
1784+ message2 = {"type": "data", "data": 2}
1785+ self.mstore.add(message1)
1786+ self.mstore.add(message2)
1787+
1788+ self.transport.next_expected_sequence = 100
1789+ self.exchanger.exchange() # Resync
1790+ self.exchanger.exchange() # Resend
1791+
1792+ # Confirm messages is not empty which was the original bug
1793+ last_messages = self.transport.payloads[-1]["messages"]
1794+ self.assertTrue(last_messages)
1795+
1796+ # Confirm earlier messages are not resent
1797+ self.assertNotIn(message0["data"],
1798+ [m["data"] for m in last_messages])
1799+
1800+ # Confirm contents of payload
1801+ self.assertEqual([message1, message2], last_messages)
1802+
1803+ def test_resync_when_next_expected_too_high(self):
1804+ '''
1805+ When next expected sequence received from the server is too high, then
1806+ a resynchronize should happen
1807+ '''
1808+
1809+ self.mstore.set_accepted_types(["empty", "resynchronize"])
1810+ self.mstore.add({"type": "empty"})
1811+ self.exchanger.exchange()
1812+
1813+ self.transport.next_expected_sequence = 100
1814+
1815+ self.reactor.call_on("resynchronize-clients", lambda scope=None: None)
1816+
1817+ self.exchanger.exchange()
1818+ self.assertMessage(self.mstore.get_pending_messages()[-1],
1819+ {"type": "resynchronize"})
1820+
1821 def test_start_with_urgent_exchange(self):
1822 """
1823 Immediately after registration, an urgent exchange should be scheduled.
1824@@ -1088,6 +1214,64 @@ class MessageExchangeTest(LandscapeTest):
1825 self.exchanger.exchange()
1826 self.assertEqual(b"3.2", self.mstore.get_server_api())
1827
1828+ def test_500_backoff(self):
1829+ """
1830+ If we get a server error then the exponential backoff is triggered
1831+ """
1832+ self.config.urgent_exchange_interval = 10
1833+ self.exchanger._backoff_counter._start_delay = 300
1834+ self.exchanger._backoff_counter._max_delay = 1000
1835+ self.transport.responses.append(HTTPCodeError(503, ""))
1836+ self.exchanger.schedule_exchange(urgent=True)
1837+ self.reactor.advance(50)
1838+ self.assertEqual(len(self.transport.payloads), 1)
1839+ self.reactor.advance(400)
1840+ self.assertEqual(len(self.transport.payloads), 2)
1841+
1842+ def test_429_backoff(self):
1843+ """
1844+ HTTP error 429 should also trigger backoff
1845+ """
1846+ self.config.urgent_exchange_interval = 10
1847+ self.exchanger._backoff_counter._start_delay = 300
1848+ self.exchanger._backoff_counter._max_delay = 1000
1849+ self.transport.responses.append(HTTPCodeError(429, ""))
1850+ self.exchanger.schedule_exchange(urgent=True)
1851+ self.reactor.advance(50)
1852+ self.assertEqual(len(self.transport.payloads), 1)
1853+
1854+ def test_backoff_reset_after_success(self):
1855+ """
1856+ If we get a success after a 500 error then backoff should be zero
1857+ """
1858+ self.config.urgent_exchange_interval = 10
1859+ self.exchanger._backoff_counter._start_delay = 300
1860+ self.exchanger._backoff_counter._max_delay = 1000
1861+ self.transport.responses.append(HTTPCodeError(500, ""))
1862+ self.exchanger.schedule_exchange(urgent=True)
1863+ self.reactor.advance(50)
1864+
1865+ # Confirm it's not zero after the error
1866+ self.assertTrue(self.exchanger._backoff_counter.get_random_delay())
1867+
1868+ server_message = [{"type": "type-R", "whatever": 5678}]
1869+ self.transport.responses.append(server_message)
1870+ self.exchanger.schedule_exchange(urgent=True)
1871+ self.reactor.advance(500)
1872+
1873+ # Confirm it is zero after the success
1874+ self.assertFalse(self.exchanger._backoff_counter.get_random_delay())
1875+
1876+ def test_400_no_backoff(self):
1877+ """
1878+ If we get a 400 error then the backoff should not be triggered
1879+ """
1880+ self.config.urgent_exchange_interval = 10
1881+ self.transport.responses.append(HTTPCodeError(400, ""))
1882+ self.exchanger.schedule_exchange(urgent=True)
1883+ self.reactor.advance(20)
1884+ self.assertEqual(len(self.transport.payloads), 2)
1885+
1886
1887 class AcceptedTypesMessageExchangeTest(LandscapeTest):
1888
1889diff --git a/landscape/client/broker/tests/test_registration.py b/landscape/client/broker/tests/test_registration.py
1890index 8ae8574..8ca0cb4 100644
1891--- a/landscape/client/broker/tests/test_registration.py
1892+++ b/landscape/client/broker/tests/test_registration.py
1893@@ -3,7 +3,7 @@ import logging
1894 import socket
1895 import mock
1896
1897-from twisted.python.compat import _PY3
1898+from landscape.lib.compat import _PY3
1899
1900 from landscape.client.broker.registration import RegistrationError, Identity
1901 from landscape.client.tests.helpers import LandscapeTest
1902@@ -90,7 +90,7 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
1903 and insecure ids even if no requests were sent.
1904 """
1905 self.exchanger.handle_message(
1906- {"type": "set-id", "id": "abc", "insecure-id": "def"})
1907+ {"type": b"set-id", "id": b"abc", "insecure-id": b"def"})
1908 self.assertEqual(self.identity.secure_id, "abc")
1909 self.assertEqual(self.identity.insecure_id, "def")
1910
1911@@ -101,29 +101,58 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
1912 """
1913 reactor_fire_mock = self.reactor.fire = mock.Mock()
1914 self.exchanger.handle_message(
1915- {"type": "set-id", "id": "abc", "insecure-id": "def"})
1916+ {"type": b"set-id", "id": b"abc", "insecure-id": b"def"})
1917 reactor_fire_mock.assert_any_call("registration-done")
1918
1919 def test_unknown_id(self):
1920 self.identity.secure_id = "old_id"
1921 self.identity.insecure_id = "old_id"
1922 self.mstore.set_accepted_types(["register"])
1923- self.exchanger.handle_message({"type": "unknown-id"})
1924+ self.exchanger.handle_message({"type": b"unknown-id"})
1925 self.assertEqual(self.identity.secure_id, None)
1926 self.assertEqual(self.identity.insecure_id, None)
1927
1928 def test_unknown_id_with_clone(self):
1929 """
1930 If the server reports us that we are a clone of another computer, then
1931- set our computer's title accordingly.
1932+ make sure we handle it
1933 """
1934 self.config.computer_title = "Wu"
1935 self.mstore.set_accepted_types(["register"])
1936- self.exchanger.handle_message({"type": "unknown-id", "clone-of": "Wu"})
1937- self.assertEqual("Wu (clone)", self.config.computer_title)
1938+ self.exchanger.handle_message(
1939+ {"type": b"unknown-id", "clone-of": "Wu"})
1940 self.assertIn("Client is clone of computer Wu",
1941 self.logfile.getvalue())
1942
1943+ def test_clone_secure_id_saved(self):
1944+ """
1945+ Make sure that secure id is saved when theres a clone and existing
1946+ value is cleared out
1947+ """
1948+ secure_id = "foo"
1949+ self.identity.secure_id = secure_id
1950+ self.config.computer_title = "Wu"
1951+ self.mstore.set_accepted_types(["register"])
1952+ self.exchanger.handle_message(
1953+ {"type": b"unknown-id", "clone-of": "Wu"})
1954+ self.assertEqual(self.handler._clone_secure_id, secure_id)
1955+ self.assertIsNone(self.identity.secure_id)
1956+
1957+ def test_clone_id_in_message(self):
1958+ """
1959+ Make sure that the clone id is present in the registration message
1960+ """
1961+ secure_id = "foo"
1962+ self.identity.secure_id = secure_id
1963+ self.config.computer_title = "Wu"
1964+ self.mstore.set_accepted_types(["register"])
1965+ self.mstore.set_server_api(b"3.3") # Note this is only for later api
1966+ self.exchanger.handle_message(
1967+ {"type": b"unknown-id", "clone-of": "Wu"})
1968+ self.reactor.fire("pre-exchange")
1969+ messages = self.mstore.get_pending_messages()
1970+ self.assertEqual(messages[0]["clone_secure_id"], secure_id)
1971+
1972 def test_should_register(self):
1973 self.mstore.set_accepted_types(["register"])
1974 self.config.computer_title = "Computer Title"
1975@@ -366,7 +395,7 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
1976 """
1977 reactor_fire_mock = self.reactor.fire = mock.Mock()
1978 self.exchanger.handle_message(
1979- {"type": "registration", "info": "unknown-account"})
1980+ {"type": b"registration", "info": b"unknown-account"})
1981 reactor_fire_mock.assert_called_with(
1982 "registration-failed", reason="unknown-account")
1983
1984@@ -378,7 +407,7 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
1985 """
1986 reactor_fire_mock = self.reactor.fire = mock.Mock()
1987 self.exchanger.handle_message(
1988- {"type": "registration", "info": "max-pending-computers"})
1989+ {"type": b"registration", "info": b"max-pending-computers"})
1990 reactor_fire_mock.assert_called_with(
1991 "registration-failed", reason="max-pending-computers")
1992
1993@@ -389,7 +418,7 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
1994 """
1995 reactor_fire_mock = self.reactor.fire = mock.Mock()
1996 self.exchanger.handle_message(
1997- {"type": "registration", "info": "blah-blah"})
1998+ {"type": b"registration", "info": b"blah-blah"})
1999 for name, args, kwargs in reactor_fire_mock.mock_calls:
2000 self.assertNotEquals("registration-failed", args[0])
2001
2002@@ -420,13 +449,13 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
2003
2004 # This should somehow callback the deferred.
2005 self.exchanger.handle_message(
2006- {"type": "set-id", "id": "abc", "insecure-id": "def"})
2007+ {"type": b"set-id", "id": b"abc", "insecure-id": b"def"})
2008
2009 self.assertEqual(calls, [1])
2010
2011 # Doing it again to ensure that the deferred isn't called twice.
2012 self.exchanger.handle_message(
2013- {"type": "set-id", "id": "abc", "insecure-id": "def"})
2014+ {"type": b"set-id", "id": b"abc", "insecure-id": b"def"})
2015
2016 self.assertEqual(calls, [1])
2017
2018@@ -448,7 +477,7 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
2019
2020 # This should somehow callback the deferred.
2021 self.exchanger.handle_message(
2022- {"type": "set-id", "id": "abc", "insecure-id": "def"})
2023+ {"type": b"set-id", "id": b"abc", "insecure-id": b"def"})
2024
2025 self.assertEqual(results, [None])
2026
2027@@ -473,13 +502,13 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
2028
2029 # This should somehow callback the deferred.
2030 self.exchanger.handle_message(
2031- {"type": "registration", "info": "unknown-account"})
2032+ {"type": b"registration", "info": b"unknown-account"})
2033
2034 self.assertEqual(calls, [True])
2035
2036 # Doing it again to ensure that the deferred isn't called twice.
2037 self.exchanger.handle_message(
2038- {"type": "registration", "info": "unknown-account"})
2039+ {"type": b"registration", "info": b"unknown-account"})
2040
2041 self.assertEqual(calls, [True])
2042
2043@@ -505,13 +534,13 @@ class RegistrationHandlerTest(RegistrationHandlerTestBase):
2044 d.addErrback(add_call)
2045
2046 self.exchanger.handle_message(
2047- {"type": "registration", "info": "max-pending-computers"})
2048+ {"type": b"registration", "info": b"max-pending-computers"})
2049
2050 self.assertEqual(calls, [True])
2051
2052 # Doing it again to ensure that the deferred isn't called twice.
2053 self.exchanger.handle_message(
2054- {"type": "registration", "info": "max-pending-computers"})
2055+ {"type": b"registration", "info": b"max-pending-computers"})
2056
2057 self.assertEqual(calls, [True])
2058
2059diff --git a/landscape/client/broker/tests/test_server.py b/landscape/client/broker/tests/test_server.py
2060index 5059e48..99fa65a 100644
2061--- a/landscape/client/broker/tests/test_server.py
2062+++ b/landscape/client/broker/tests/test_server.py
2063@@ -130,6 +130,25 @@ class BrokerServerTest(LandscapeTest):
2064 self.assertMessages(self.mstore.get_pending_messages(), [message])
2065 self.assertTrue(self.exchanger.is_urgent())
2066
2067+ def test_send_message_from_py27_upgrader(self):
2068+ """
2069+ If we receive release-upgrade results from a py27 release upgrader,
2070+ it gets translated to a py3-compatible message.
2071+ """
2072+ legacy_message = {
2073+ b"type": b"change-packages-result",
2074+ b"operation-id": 99,
2075+ b"result-code": 123}
2076+ self.mstore.set_accepted_types(["change-packages-result"])
2077+ self.broker.send_message(legacy_message, True)
2078+ expected = [{
2079+ "type": "change-packages-result",
2080+ "operation-id": 99,
2081+ "result-code": 123
2082+ }]
2083+ self.assertMessages(self.mstore.get_pending_messages(), expected)
2084+ self.assertTrue(self.exchanger.is_urgent())
2085+
2086 def test_is_pending(self):
2087 """
2088 The L{BrokerServer.is_pending} method indicates if a message with
2089diff --git a/landscape/client/broker/tests/test_store.py b/landscape/client/broker/tests/test_store.py
2090index efa64d4..208c75c 100644
2091--- a/landscape/client/broker/tests/test_store.py
2092+++ b/landscape/client/broker/tests/test_store.py
2093@@ -149,7 +149,7 @@ class MessageStoreTest(LandscapeTest):
2094 for i in range(10):
2095 self.store.add(dict(type="data", data=intToBytes(i)))
2096 il = [m["data"] for m in self.store.get_pending_messages(5)]
2097- self.assertEqual(il, [intToBytes(i) for i in[0, 1, 2, 3, 4]])
2098+ self.assertEqual(il, [intToBytes(i) for i in [0, 1, 2, 3, 4]])
2099
2100 def test_offset(self):
2101 self.store.set_pending_offset(5)
2102diff --git a/landscape/client/broker/tests/test_transport.py b/landscape/client/broker/tests/test_transport.py
2103index 537c36d..87106a1 100644
2104--- a/landscape/client/broker/tests/test_transport.py
2105+++ b/landscape/client/broker/tests/test_transport.py
2106@@ -16,7 +16,7 @@ from twisted.internet.threads import deferToThread
2107
2108
2109 def sibpath(path):
2110- return os.path.join(os.path.dirname(__file__), path)
2111+ return os.path.abspath(os.path.join(os.path.dirname(__file__), path))
2112
2113
2114 PRIVKEY = sibpath("private.ssl")
2115diff --git a/landscape/client/broker/transport.py b/landscape/client/broker/transport.py
2116index ed080cd..27b70fb 100644
2117--- a/landscape/client/broker/transport.py
2118+++ b/landscape/client/broker/transport.py
2119@@ -6,7 +6,7 @@ import uuid
2120
2121 import pycurl
2122
2123-from twisted.python.compat import unicode, _PY3
2124+from landscape.lib.compat import unicode, _PY3
2125
2126 from landscape.lib import bpickle
2127 from landscape.lib.fetch import fetch
2128diff --git a/landscape/client/configuration.py b/landscape/client/configuration.py
2129index e08a865..e06dd26 100644
2130--- a/landscape/client/configuration.py
2131+++ b/landscape/client/configuration.py
2132@@ -7,14 +7,15 @@ for the C{landscape-config} script.
2133 from __future__ import print_function
2134
2135 from functools import partial
2136-import base64
2137 import getpass
2138 import io
2139 import os
2140 import pwd
2141 import sys
2142+import textwrap
2143
2144 from landscape.lib.compat import input
2145+from landscape.lib import base64
2146
2147 from landscape.lib.tag import is_valid_tag
2148
2149@@ -33,6 +34,9 @@ from landscape.client.broker.registration import Identity
2150 from landscape.client.broker.service import BrokerService
2151
2152
2153+EXIT_NOT_REGISTERED = 5
2154+
2155+
2156 class ConfigurationError(Exception):
2157 """Raised when required configuration values are missing."""
2158
2159@@ -192,6 +196,11 @@ class LandscapeSetupConfiguration(BrokerConfiguration):
2160 parser.add_option("--init", action="store_true", default=False,
2161 help="Set up the client directories structure "
2162 "and exit.")
2163+ parser.add_option("--is-registered", action="store_true",
2164+ help="Exit with code 0 (success) if client is "
2165+ "registered else returns {}. Displays "
2166+ "registration info."
2167+ .format(EXIT_NOT_REGISTERED))
2168 return parser
2169
2170
2171@@ -513,7 +522,7 @@ def decode_base64_ssl_public_certificate(config):
2172 # WARNING: ssl_public_certificate is misnamed, it's not the key of the
2173 # certificate, but the actual certificate itself.
2174 if config.ssl_public_key and config.ssl_public_key.startswith("base64:"):
2175- decoded_cert = base64.decodestring(
2176+ decoded_cert = base64.decodebytes(
2177 config.ssl_public_key[7:].encode("ascii"))
2178 config.ssl_public_key = store_public_key_data(
2179 config, decoded_cert)
2180@@ -759,6 +768,26 @@ def is_registered(config):
2181 return bool(identity.secure_id)
2182
2183
2184+def registration_info_text(config, registration_status):
2185+ '''
2186+ A simple output displaying whether the client is registered or not, the
2187+ account name, and config and data paths
2188+ '''
2189+
2190+ config_path = os.path.abspath(config._config_filename)
2191+
2192+ text = textwrap.dedent("""
2193+ Registered: {}
2194+ Config Path: {}
2195+ Data Path {}"""
2196+ .format(registration_status, config_path,
2197+ config.data_path))
2198+ if registration_status:
2199+ text += '\nAccount Name: {}'.format(config.account_name)
2200+
2201+ return text
2202+
2203+
2204 def main(args, print=print):
2205 """Interact with the user and the server to set up client configuration."""
2206
2207@@ -769,6 +798,18 @@ def main(args, print=print):
2208 print_text(str(error), error=True)
2209 sys.exit(1)
2210
2211+ if config.is_registered:
2212+
2213+ registration_status = is_registered(config)
2214+
2215+ info_text = registration_info_text(config, registration_status)
2216+ print(info_text)
2217+
2218+ if registration_status:
2219+ sys.exit(0)
2220+ else:
2221+ sys.exit(EXIT_NOT_REGISTERED)
2222+
2223 if os.getuid() != 0:
2224 sys.exit("landscape-config must be run as root.")
2225
2226diff --git a/landscape/client/lockfile.py b/landscape/client/lockfile.py
2227new file mode 100644
2228index 0000000..aa0f916
2229--- /dev/null
2230+++ b/landscape/client/lockfile.py
2231@@ -0,0 +1,52 @@
2232+import os
2233+
2234+from twisted.python import lockfile
2235+
2236+
2237+def patch_lockfile():
2238+ if lockfile.FilesystemLock is PatchedFilesystemLock:
2239+ return
2240+ lockfile.FilesystemLock = PatchedFilesystemLock
2241+
2242+
2243+class PatchedFilesystemLock(lockfile.FilesystemLock):
2244+ """
2245+ Patched Twisted's FilesystemLock.lock to handle PermissionError
2246+ when trying to lock.
2247+ """
2248+
2249+ def lock(self):
2250+ # XXX Twisted assumes PIDs don't get reused, which is incorrect.
2251+ # As such, we pre-check that any existing lock file isn't
2252+ # associated to a live process, and that any associated
2253+ # process is from landscape. Otherwise, clean up the lock file,
2254+ # considering it to be locked to a recycled PID.
2255+ #
2256+ # Although looking for the process name may seem fragile, it's the
2257+ # most acurate info we have since:
2258+ # * some process run as root, so the UID is not a reference
2259+ # * process may not be spawned by systemd, so cgroups are not reliable
2260+ # * python executable is not a reference
2261+ clean = True
2262+ try:
2263+ pid = os.readlink(self.name)
2264+ ps_name = get_process_name(int(pid))
2265+ if not ps_name.startswith("landscape"):
2266+ os.remove(self.name)
2267+ clean = False
2268+ except Exception:
2269+ # We can't figure the lock state, let FilesystemLock figure it
2270+ # out normally.
2271+ pass
2272+
2273+ result = super(PatchedFilesystemLock, self).lock()
2274+ self.clean = self.clean and clean
2275+ return result
2276+
2277+
2278+def get_process_name(pid):
2279+ """Return a process name from a pid."""
2280+ stat_path = "/proc/{}/stat".format(pid)
2281+ with open(stat_path) as stat_file:
2282+ stat = stat_file.read()
2283+ return stat.partition("(")[2].rpartition(")")[0]
2284diff --git a/landscape/client/manager/aptsources.py b/landscape/client/manager/aptsources.py
2285index 649569b..866c6ba 100644
2286--- a/landscape/client/manager/aptsources.py
2287+++ b/landscape/client/manager/aptsources.py
2288@@ -4,6 +4,7 @@ import pwd
2289 import grp
2290 import shutil
2291 import tempfile
2292+import uuid
2293
2294 from twisted.internet.defer import succeed
2295
2296@@ -22,6 +23,7 @@ class AptSources(ManagerPlugin):
2297
2298 SOURCES_LIST = "/etc/apt/sources.list"
2299 SOURCES_LIST_D = "/etc/apt/sources.list.d"
2300+ TRUSTED_GPG_D = "/etc/apt/trusted.gpg.d"
2301
2302 def register(self, registry):
2303 super(AptSources, self).register(registry)
2304@@ -83,16 +85,12 @@ class AptSources(ManagerPlugin):
2305 "-----END PGP PUBLIC KEY BLOCK-----"]}
2306 """
2307 deferred = succeed(None)
2308+ prefix = 'landscape-server-'
2309 for key in message["gpg-keys"]:
2310- fd, path = tempfile.mkstemp()
2311- os.close(fd)
2312- with open(path, "w") as key_file:
2313+ filename = prefix + str(uuid.uuid4()) + '.asc'
2314+ key_path = os.path.join(self.TRUSTED_GPG_D, filename)
2315+ with open(key_path, "w") as key_file:
2316 key_file.write(key)
2317- deferred.addCallback(
2318- lambda ignore, path=path:
2319- self._run_process("/usr/bin/apt-key", ["add", path]))
2320- deferred.addCallback(self._handle_process_error)
2321- deferred.addBoth(self._remove_and_continue, path)
2322 deferred.addErrback(self._handle_process_failure)
2323 deferred.addCallback(self._handle_sources, message["sources"])
2324 return self.call_with_operation_result(message, lambda: deferred)
2325diff --git a/landscape/client/manager/config.py b/landscape/client/manager/config.py
2326index 1461767..bfc0a5d 100644
2327--- a/landscape/client/manager/config.py
2328+++ b/landscape/client/manager/config.py
2329@@ -30,6 +30,14 @@ class ManagerConfiguration(Configuration):
2330 help="Comma-delimited list of usernames that scripts"
2331 " may be run as. Default is to allow all "
2332 "users.")
2333+ parser.add_option("--script-output-limit",
2334+ metavar="SCRIPT_OUTPUT_LIMIT",
2335+ type="int", default=512,
2336+ help="Maximum allowed output size that scripts"
2337+ " can send. "
2338+ "Script output will be truncated at that limit."
2339+ " Default is 512 (kB)")
2340+
2341 return parser
2342
2343 @property
2344diff --git a/landscape/client/manager/keystonetoken.py b/landscape/client/manager/keystonetoken.py
2345index f79d4bc..687f5c7 100644
2346--- a/landscape/client/manager/keystonetoken.py
2347+++ b/landscape/client/manager/keystonetoken.py
2348@@ -1,7 +1,7 @@
2349 import os
2350 import logging
2351
2352-from twisted.python.compat import _PY3
2353+from landscape.lib.compat import _PY3
2354
2355 from landscape.lib.compat import ConfigParser, NoOptionError
2356 from landscape.client.monitor.plugin import DataWatcher
2357diff --git a/landscape/client/manager/scriptexecution.py b/landscape/client/manager/scriptexecution.py
2358index 4481e17..b20b58a 100644
2359--- a/landscape/client/manager/scriptexecution.py
2360+++ b/landscape/client/manager/scriptexecution.py
2361@@ -115,7 +115,8 @@ class ScriptRunnerMixin(object):
2362 }
2363
2364 pp = ProcessAccumulationProtocol(
2365- self.registry.reactor, self.size_limit, self.truncation_indicator)
2366+ self.registry.reactor, self.registry.config.script_output_limit,
2367+ self.truncation_indicator)
2368 args = (filename,)
2369 self.process_factory.spawnProcess(
2370 pp, filename, args=args, uid=uid, gid=gid, path=path, env=env)
2371@@ -127,11 +128,8 @@ class ScriptRunnerMixin(object):
2372 class ScriptExecutionPlugin(ManagerPlugin, ScriptRunnerMixin):
2373 """A plugin which allows execution of arbitrary shell scripts.
2374
2375- @ivar size_limit: The number of bytes at which to truncate process output.
2376 """
2377
2378- size_limit = 500000
2379-
2380 def register(self, registry):
2381 super(ScriptExecutionPlugin, self).register(registry)
2382 registry.register_message(
2383@@ -316,7 +314,7 @@ class ProcessAccumulationProtocol(ProcessProtocol):
2384 self._size = 0
2385 self.result_deferred = Deferred()
2386 self._cancelled = False
2387- self.size_limit = size_limit
2388+ self.size_limit = size_limit * 1024
2389 self._truncation_indicator = truncation_indicator.encode("utf-8")
2390 self._truncation_offset = len(self._truncation_indicator)
2391 self._truncated_size_limit = self.size_limit - self._truncation_offset
2392diff --git a/landscape/client/manager/service.py b/landscape/client/manager/service.py
2393index b95f4eb..ef543d6 100644
2394--- a/landscape/client/manager/service.py
2395+++ b/landscape/client/manager/service.py
2396@@ -54,8 +54,9 @@ class ManagerService(LandscapeService):
2397 def stopService(self):
2398 """Stop the manager and close the connection with the broker."""
2399 self.connector.disconnect()
2400- self.publisher.stop()
2401+ deferred = self.publisher.stop()
2402 super(ManagerService, self).stopService()
2403+ return deferred
2404
2405
2406 def run(args):
2407diff --git a/landscape/client/manager/tests/test_aptsources.py b/landscape/client/manager/tests/test_aptsources.py
2408index b63d4db..a4d2cd8 100644
2409--- a/landscape/client/manager/tests/test_aptsources.py
2410+++ b/landscape/client/manager/tests/test_aptsources.py
2411@@ -7,7 +7,6 @@ from twisted.internet.defer import Deferred, succeed
2412 from landscape.client.manager.aptsources import AptSources
2413 from landscape.client.manager.plugin import SUCCEEDED, FAILED
2414
2415-from landscape.lib.twisted_util import gather_results, SignalError
2416 from landscape.client.tests.helpers import LandscapeTest, ManagerHelper
2417 from landscape.client.package.reporter import find_reporter_command
2418
2419@@ -272,210 +271,23 @@ class AptSourcesTests(LandscapeTest):
2420 C{AptSources} runs a process with apt-key for every keys in the
2421 message.
2422 """
2423- deferred = Deferred()
2424-
2425- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2426- self.assertEqual("/usr/bin/apt-key", command)
2427- self.assertEqual("add", args[0])
2428- filename = args[1]
2429- with open(filename) as file:
2430- result = file.read()
2431- self.assertEqual("Some key content", result)
2432- deferred.callback(("ok", "", 0))
2433- return deferred
2434-
2435- self.sourceslist._run_process = _run_process
2436-
2437- self.manager.dispatch_message(
2438- {"type": "apt-sources-replace", "sources": [],
2439- "gpg-keys": ["Some key content"], "operation-id": 1})
2440-
2441- return deferred
2442-
2443- def test_import_delete_temporary_files(self):
2444- """
2445- The files created to be imported by C{apt-key} are removed after the
2446- import.
2447- """
2448- deferred = Deferred()
2449- filenames = []
2450-
2451- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2452- if not filenames:
2453- filenames.append(args[1])
2454- deferred.callback(("ok", "", 0))
2455- return deferred
2456-
2457- self.sourceslist._run_process = _run_process
2458-
2459- self.manager.dispatch_message(
2460- {"type": "apt-sources-replace", "sources": [],
2461- "gpg-keys": ["Some key content"], "operation-id": 1})
2462-
2463- self.assertFalse(os.path.exists(filenames[0]))
2464-
2465- return deferred
2466-
2467- def test_failed_import_delete_temporary_files(self):
2468- """
2469- The files created to be imported by C{apt-key} are removed after the
2470- import, even if there is a failure.
2471- """
2472- deferred = Deferred()
2473- filenames = []
2474-
2475- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2476- filenames.append(args[1])
2477- deferred.callback(("error", "", 1))
2478- return deferred
2479
2480- self.sourceslist._run_process = _run_process
2481+ self.sourceslist.TRUSTED_GPG_D = self.makeDir()
2482
2483+ gpg_keys = ["key1", "key2"]
2484 self.manager.dispatch_message(
2485 {"type": "apt-sources-replace", "sources": [],
2486- "gpg-keys": ["Some key content"], "operation-id": 1})
2487-
2488- self.assertFalse(os.path.exists(filenames[0]))
2489-
2490- return deferred
2491-
2492- def test_failed_import_reported(self):
2493- """
2494- If the C{apt-key} command failed for some reasons, the output of the
2495- command is reported and the activity fails.
2496- """
2497- deferred = Deferred()
2498-
2499- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2500- deferred.callback(("nok", "some error", 1))
2501- return deferred
2502-
2503- self.sourceslist._run_process = _run_process
2504-
2505- self.manager.dispatch_message(
2506- {"type": "apt-sources-replace", "sources": [], "gpg-keys": ["key"],
2507+ "gpg-keys": gpg_keys,
2508 "operation-id": 1})
2509
2510- service = self.broker_service
2511- msg = "ProcessError: nok\nsome error"
2512- self.assertMessages(service.message_store.get_pending_messages(),
2513- [{"type": "operation-result",
2514- "result-text": msg, "status": FAILED,
2515- "operation-id": 1}])
2516- return deferred
2517+ keys = []
2518+ gpg_dirpath = self.sourceslist.TRUSTED_GPG_D
2519+ for filename in os.listdir(gpg_dirpath):
2520+ filepath = os.path.join(gpg_dirpath, filename)
2521+ with open(filepath, 'r') as fh:
2522+ keys.append(fh.read())
2523
2524- def test_signaled_import_reported(self):
2525- """
2526- If the C{apt-key} fails with a signal, the output of the command is
2527- reported and the activity fails.
2528- """
2529- deferred = Deferred()
2530-
2531- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2532- deferred.errback(SignalError("nok", "some error", 1))
2533- return deferred
2534-
2535- self.sourceslist._run_process = _run_process
2536-
2537- self.manager.dispatch_message(
2538- {"type": "apt-sources-replace", "sources": [], "gpg-keys": ["key"],
2539- "operation-id": 1})
2540-
2541- service = self.broker_service
2542- msg = "ProcessError: nok\nsome error"
2543- self.assertMessages(service.message_store.get_pending_messages(),
2544- [{"type": "operation-result",
2545- "result-text": msg, "status": FAILED,
2546- "operation-id": 1}])
2547- return deferred
2548-
2549- def test_failed_import_no_changes(self):
2550- """
2551- If the C{apt-key} command failed for some reasons, the current
2552- repositories aren't changed.
2553- """
2554- deferred = Deferred()
2555-
2556- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2557- deferred.callback(("nok", "some error", 1))
2558- return deferred
2559-
2560- self.sourceslist._run_process = _run_process
2561-
2562- with open(self.sourceslist.SOURCES_LIST, "w") as sources:
2563- sources.write("oki\n\ndoki\n#comment\n")
2564-
2565- self.manager.dispatch_message(
2566- {"type": "apt-sources-replace", "sources": [], "gpg-keys": ["key"],
2567- "operation-id": 1})
2568-
2569- with open(self.sourceslist.SOURCES_LIST) as sources_list:
2570- result = sources_list.read()
2571-
2572- self.assertEqual("oki\n\ndoki\n#comment\n", result)
2573-
2574- return deferred
2575-
2576- def test_multiple_import_sequential(self):
2577- """
2578- If multiple keys are specified, the imports run sequentially, not in
2579- parallel.
2580- """
2581- deferred1 = Deferred()
2582- deferred2 = Deferred()
2583- deferreds = [deferred1, deferred2]
2584-
2585- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2586- if not deferreds:
2587- return succeed(None)
2588- return deferreds.pop(0)
2589-
2590- self.sourceslist._run_process = _run_process
2591-
2592- self.manager.dispatch_message(
2593- {"type": "apt-sources-replace", "sources": [],
2594- "gpg-keys": ["key1", "key2"], "operation-id": 1})
2595-
2596- self.assertEqual(1, len(deferreds))
2597- deferred1.callback(("ok", "", 0))
2598-
2599- self.assertEqual(0, len(deferreds))
2600- deferred2.callback(("ok", "", 0))
2601-
2602- service = self.broker_service
2603- self.assertMessages(service.message_store.get_pending_messages(),
2604- [{"type": "operation-result",
2605- "status": SUCCEEDED, "operation-id": 1}])
2606- return gather_results(deferreds)
2607-
2608- def test_multiple_import_failure(self):
2609- """
2610- If multiple keys are specified, and that the first one fails, the error
2611- is correctly reported.
2612- """
2613- deferred1 = Deferred()
2614- deferred2 = Deferred()
2615- deferreds = [deferred1, deferred2]
2616-
2617- def _run_process(command, args, env={}, path=None, uid=None, gid=None):
2618- return deferreds.pop(0)
2619-
2620- self.sourceslist._run_process = _run_process
2621-
2622- self.manager.dispatch_message(
2623- {"type": "apt-sources-replace", "sources": [],
2624- "gpg-keys": ["key1", "key2"], "operation-id": 1})
2625-
2626- deferred1.callback(("error", "", 1))
2627- deferred2.callback(("error", "", 1))
2628-
2629- msg = "ProcessError: error\n"
2630- service = self.broker_service
2631- self.assertMessages(service.message_store.get_pending_messages(),
2632- [{"type": "operation-result",
2633- "result-text": msg, "status": FAILED,
2634- "operation-id": 1}])
2635- return gather_results(deferreds)
2636+ self.assertCountEqual(keys, gpg_keys)
2637
2638 def test_run_reporter(self):
2639 """
2640diff --git a/landscape/client/manager/tests/test_processkiller.py b/landscape/client/manager/tests/test_processkiller.py
2641index 75ef833..ae45b0a 100644
2642--- a/landscape/client/manager/tests/test_processkiller.py
2643+++ b/landscape/client/manager/tests/test_processkiller.py
2644@@ -14,7 +14,7 @@ from landscape.client.manager.processkiller import (
2645
2646
2647 def get_active_process():
2648- return subprocess.Popen(["python", "-c", "raw_input()"],
2649+ return subprocess.Popen(["python3", "-c", "input()"],
2650 stdin=subprocess.PIPE,
2651 stdout=subprocess.PIPE,
2652 stderr=subprocess.PIPE)
2653diff --git a/landscape/client/manager/tests/test_scriptexecution.py b/landscape/client/manager/tests/test_scriptexecution.py
2654index 2f21949..db3ef12 100644
2655--- a/landscape/client/manager/tests/test_scriptexecution.py
2656+++ b/landscape/client/manager/tests/test_scriptexecution.py
2657@@ -70,7 +70,7 @@ class RunScriptTests(LandscapeTest):
2658
2659 def test_other_interpreter(self):
2660 """Non-shell interpreters can be specified."""
2661- result = self.plugin.run_script("/usr/bin/python", "print 'hi'")
2662+ result = self.plugin.run_script("/usr/bin/python3", "print('hi')")
2663 result.addCallback(self.assertEqual, "hi\n")
2664 return result
2665
2666@@ -492,18 +492,18 @@ class RunScriptTests(LandscapeTest):
2667 """Data returned from the command is limited."""
2668 factory = StubProcessFactory()
2669 self.plugin.process_factory = factory
2670- self.plugin.size_limit = 100
2671+ self.manager.config.script_output_limit = 1
2672 result = self.plugin.run_script("/bin/sh", "")
2673
2674 # Ultimately we assert that the resulting output is limited to
2675- # 100 bytes and indicates its truncation.
2676+ # 1024 bytes and indicates its truncation.
2677 result.addCallback(self.assertEqual,
2678- ("x" * 79) + "\n**OUTPUT TRUNCATED**")
2679+ ("x" * (1024 - 21)) + "\n**OUTPUT TRUNCATED**")
2680
2681 protocol = factory.spawns[0][0]
2682
2683- # Push 200 bytes of output, so we trigger truncation.
2684- protocol.childDataReceived(1, b"x" * 200)
2685+ # Push 2kB of output, so we trigger truncation.
2686+ protocol.childDataReceived(1, b"x" * (2*1024))
2687
2688 for fd in (0, 1, 2):
2689 protocol.childConnectionLost(fd)
2690@@ -515,19 +515,19 @@ class RunScriptTests(LandscapeTest):
2691 """After truncation, no further output is recorded."""
2692 factory = StubProcessFactory()
2693 self.plugin.process_factory = factory
2694- self.plugin.size_limit = 100
2695+ self.manager.config.script_output_limit = 1
2696 result = self.plugin.run_script("/bin/sh", "")
2697
2698 # Ultimately we assert that the resulting output is limited to
2699- # 100 bytes and indicates its truncation.
2700+ # 1024 bytes and indicates its truncation.
2701 result.addCallback(self.assertEqual,
2702- ("x" * 79) + "\n**OUTPUT TRUNCATED**")
2703+ ("x" * (1024 - 21)) + "\n**OUTPUT TRUNCATED**")
2704 protocol = factory.spawns[0][0]
2705
2706- # Push 200 bytes of output, so we trigger truncation.
2707- protocol.childDataReceived(1, b"x" * 200)
2708- # Push 200 bytes more
2709- protocol.childDataReceived(1, b"x" * 200)
2710+ # Push 1024 bytes of output, so we trigger truncation.
2711+ protocol.childDataReceived(1, b"x" * 1024)
2712+ # Push 1024 bytes more
2713+ protocol.childDataReceived(1, b"x" * 1024)
2714
2715 for fd in (0, 1, 2):
2716 protocol.childConnectionLost(fd)
2717diff --git a/landscape/client/monitor/__init__.py b/landscape/client/monitor/__init__.py
2718index d8678a3..bb6b536 100644
2719--- a/landscape/client/monitor/__init__.py
2720+++ b/landscape/client/monitor/__init__.py
2721@@ -1,4 +1,4 @@
2722 """
2723 The monitor extracts data about the local machine and sends it in
2724-messages to the Landcsape server via the broker.
2725+messages to the Landscape server via the broker.
2726 """
2727diff --git a/landscape/client/monitor/computertags.py b/landscape/client/monitor/computertags.py
2728new file mode 100644
2729index 0000000..1429367
2730--- /dev/null
2731+++ b/landscape/client/monitor/computertags.py
2732@@ -0,0 +1,29 @@
2733+import logging
2734+import sys
2735+
2736+from landscape.client.broker.config import BrokerConfiguration
2737+from landscape.client.monitor.plugin import DataWatcher
2738+from landscape.lib.tag import is_valid_tag_list
2739+
2740+
2741+class ComputerTags(DataWatcher):
2742+ """Plugin watches config file for changes in computer tags"""
2743+
2744+ persist_name = "computer-tags"
2745+ message_type = "computer-tags"
2746+ message_key = "tags"
2747+ run_interval = 3600 # Every hour only when data changed
2748+ run_immediately = True
2749+
2750+ def __init__(self, args=sys.argv):
2751+ super(ComputerTags, self).__init__()
2752+ self.args = args # Defined to specify args in unit tests
2753+
2754+ def get_data(self):
2755+ config = BrokerConfiguration()
2756+ config.load(self.args) # Load the default or specified config
2757+ tags = config.tags
2758+ if not is_valid_tag_list(tags):
2759+ tags = None
2760+ logging.warning("Invalid tags provided for computer-tags message.")
2761+ return tags
2762diff --git a/landscape/client/monitor/config.py b/landscape/client/monitor/config.py
2763index 2525157..19f6aa4 100644
2764--- a/landscape/client/monitor/config.py
2765+++ b/landscape/client/monitor/config.py
2766@@ -6,7 +6,7 @@ ALL_PLUGINS = ["ActiveProcessInfo", "ComputerInfo",
2767 "Temperature", "PackageMonitor", "UserMonitor",
2768 "RebootRequired", "AptPreferences", "NetworkActivity",
2769 "NetworkDevice", "UpdateManager", "CPUUsage", "SwiftUsage",
2770- "CephUsage"]
2771+ "CephUsage", "ComputerTags", "UbuntuProInfo"]
2772
2773
2774 class MonitorConfiguration(Configuration):
2775diff --git a/landscape/client/monitor/processorinfo.py b/landscape/client/monitor/processorinfo.py
2776index c129707..aaa15e3 100644
2777--- a/landscape/client/monitor/processorinfo.py
2778+++ b/landscape/client/monitor/processorinfo.py
2779@@ -87,7 +87,8 @@ class ProcessorInfo(MonitorPlugin):
2780 dirty = True
2781
2782 if dirty:
2783- logging.info("Queueing message with updated processor info.")
2784+ logging.info("Queueing updated processor info. Contents:")
2785+ logging.info(message)
2786 self.registry.broker.send_message(
2787 message, self._session_id, urgent=urgent)
2788
2789@@ -313,8 +314,47 @@ class S390XMessageFactory:
2790 return processors
2791
2792
2793+class RISCVMessageFactory:
2794+ """Factory for risc-v-based processors provides processor information.
2795+
2796+ @param source_filename: The file name of the data source.
2797+ """
2798+
2799+ def __init__(self, source_filename):
2800+ self._source_filename = source_filename
2801+
2802+ def create_message(self):
2803+ """Returns a list containing information about each processor."""
2804+ processors = []
2805+ file = open(self._source_filename)
2806+ logging.info("Entered RISCVMessageFactory")
2807+
2808+ try:
2809+ current = None
2810+
2811+ for line in file:
2812+ parts = line.split(":", 1)
2813+ key = parts[0].strip()
2814+
2815+ if key == "processor":
2816+ current = {"processor-id": int(parts[1].strip())}
2817+ processors.append(current)
2818+ elif key == "isa":
2819+ current["vendor"] = parts[1].strip()
2820+ elif key == "uarch":
2821+ current["model"] = parts[1].strip()
2822+
2823+ finally:
2824+ file.close()
2825+
2826+ logging.info("RISC-V processor info collected:")
2827+ logging.info(processors)
2828+ return processors
2829+
2830+
2831 message_factories = [("(arm*|aarch64)", ARMMessageFactory),
2832 ("ppc(64)?", PowerPCMessageFactory),
2833 ("sparc[64]", SparcMessageFactory),
2834 ("i[3-7]86|x86_64", X86MessageFactory),
2835- ("s390x", S390XMessageFactory)]
2836+ ("s390x", S390XMessageFactory),
2837+ ("riscv64", RISCVMessageFactory)]
2838diff --git a/landscape/client/monitor/service.py b/landscape/client/monitor/service.py
2839index 77d6f20..6b9c63a 100644
2840--- a/landscape/client/monitor/service.py
2841+++ b/landscape/client/monitor/service.py
2842@@ -56,10 +56,11 @@ class MonitorService(LandscapeService):
2843 The monitor is flushed to ensure that things like persist databases
2844 get saved to disk.
2845 """
2846- self.publisher.stop()
2847+ deferred = self.publisher.stop()
2848 self.monitor.flush()
2849 self.connector.disconnect()
2850 super(MonitorService, self).stopService()
2851+ return deferred
2852
2853
2854 def run(args):
2855diff --git a/landscape/client/monitor/tests/test_computertags.py b/landscape/client/monitor/tests/test_computertags.py
2856new file mode 100644
2857index 0000000..7f52943
2858--- /dev/null
2859+++ b/landscape/client/monitor/tests/test_computertags.py
2860@@ -0,0 +1,64 @@
2861+from landscape.client.monitor.computertags import ComputerTags
2862+from landscape.client.tests.helpers import MonitorHelper, LandscapeTest
2863+
2864+
2865+class ComputerTagsTest(LandscapeTest):
2866+
2867+ helpers = [MonitorHelper]
2868+
2869+ def setUp(self):
2870+ super(ComputerTagsTest, self).setUp()
2871+ test_sys_args = ['hello.py']
2872+ self.plugin = ComputerTags(args=test_sys_args)
2873+ self.monitor.add(self.plugin)
2874+
2875+ def test_tags_are_read(self):
2876+ """
2877+ Tags are read from the default config path
2878+ """
2879+ tags = 'check,linode,profile-test'
2880+ file_text = "[client]\ntags = {}".format(tags)
2881+ config_filename = self.config.default_config_filenames[0]
2882+ self.makeFile(file_text, path=config_filename)
2883+ self.assertEqual(self.plugin.get_data(), tags)
2884+
2885+ def test_tags_are_read_from_args_path(self):
2886+ """
2887+ Tags are read from path specified in command line args
2888+ """
2889+ tags = 'check,linode,profile-test'
2890+ file_text = "[client]\ntags = {}".format(tags)
2891+ filename = self.makeFile(file_text)
2892+ test_sys_args = ['hello.py', '--config', filename]
2893+ self.plugin.args = test_sys_args
2894+ self.assertEqual(self.plugin.get_data(), tags)
2895+
2896+ def test_tags_message_sent(self):
2897+ """
2898+ Tags message is sent correctly
2899+ """
2900+ tags = 'check,linode,profile-test'
2901+ file_text = "[client]\ntags = {}".format(tags)
2902+ config_filename = self.config.default_config_filenames[0]
2903+ self.makeFile(file_text, path=config_filename)
2904+
2905+ self.mstore.set_accepted_types(["computer-tags"])
2906+ self.plugin.exchange()
2907+ messages = self.mstore.get_pending_messages()
2908+ self.assertEqual(messages[0]['tags'], tags)
2909+
2910+ def test_invalid_tags(self):
2911+ """
2912+ If invalid tag detected then message contents should be None
2913+ """
2914+ tags = 'check,lin ode'
2915+ file_text = "[client]\ntags = {}".format(tags)
2916+ config_filename = self.config.default_config_filenames[0]
2917+ self.makeFile(file_text, path=config_filename)
2918+ self.assertEqual(self.plugin.get_data(), None)
2919+
2920+ def test_empty_tags(self):
2921+ """
2922+ Makes sure no errors when tags section is empty
2923+ """
2924+ self.assertEqual(self.plugin.get_data(), None)
2925diff --git a/landscape/client/monitor/tests/test_ubuntuproinfo.py b/landscape/client/monitor/tests/test_ubuntuproinfo.py
2926new file mode 100644
2927index 0000000..ece255c
2928--- /dev/null
2929+++ b/landscape/client/monitor/tests/test_ubuntuproinfo.py
2930@@ -0,0 +1,47 @@
2931+from unittest import mock
2932+
2933+from landscape.client.monitor.ubuntuproinfo import UbuntuProInfo
2934+from landscape.client.tests.helpers import LandscapeTest, MonitorHelper
2935+
2936+
2937+class UbuntuProInfoTest(LandscapeTest):
2938+ """Ubuntu Pro info plugin tests."""
2939+
2940+ helpers = [MonitorHelper]
2941+
2942+ def setUp(self):
2943+ super(UbuntuProInfoTest, self).setUp()
2944+ self.mstore.set_accepted_types(["ubuntu-pro-info"])
2945+
2946+ def test_ubuntu_pro_info(self):
2947+ """Tests calling `ua status`."""
2948+ plugin = UbuntuProInfo()
2949+ self.monitor.add(plugin)
2950+
2951+ with mock.patch("subprocess.run") as run_mock:
2952+ run_mock.return_value = mock.Mock(
2953+ stdout="\"This is a test\"",
2954+ )
2955+ plugin.exchange()
2956+
2957+ messages = self.mstore.get_pending_messages()
2958+ run_mock.assert_called_once()
2959+ self.assertTrue(len(messages) > 0)
2960+ self.assertTrue("ubuntu-pro-info" in messages[0])
2961+ self.assertEqual(messages[0]["ubuntu-pro-info"],
2962+ "\"This is a test\"")
2963+
2964+ def test_ubuntu_pro_info_no_ua(self):
2965+ """Tests calling `ua status` when it is not installed."""
2966+ plugin = UbuntuProInfo()
2967+ self.monitor.add(plugin)
2968+
2969+ with mock.patch("subprocess.run") as run_mock:
2970+ run_mock.side_effect = FileNotFoundError()
2971+ plugin.exchange()
2972+
2973+ messages = self.mstore.get_pending_messages()
2974+ run_mock.assert_called_once()
2975+ self.assertTrue(len(messages) > 0)
2976+ self.assertTrue("ubuntu-pro-info" in messages[0])
2977+ self.assertIn("errors", messages[0]["ubuntu-pro-info"])
2978diff --git a/landscape/client/monitor/ubuntuproinfo.py b/landscape/client/monitor/ubuntuproinfo.py
2979new file mode 100644
2980index 0000000..4f78999
2981--- /dev/null
2982+++ b/landscape/client/monitor/ubuntuproinfo.py
2983@@ -0,0 +1,47 @@
2984+import json
2985+import subprocess
2986+
2987+from landscape.client.monitor.plugin import DataWatcher
2988+
2989+
2990+class UbuntuProInfo(DataWatcher):
2991+ """
2992+ Plugin that captures and reports Ubuntu Pro registration
2993+ information.
2994+
2995+ We use the `ua` CLI with output formatted as JSON. This is sent
2996+ as-is and parsed by Landscape Server because the JSON content is
2997+ considered "Experimental" and we don't want to have to change in
2998+ both Client and Server in the event that the format changes.
2999+ """
3000+
3001+ run_interval = 900 # 15 minutes
3002+ message_type = "ubuntu-pro-info"
3003+ message_key = message_type
3004+ persist_name = message_type
3005+ scope = "ubuntu-pro"
3006+ run_immediately = True
3007+
3008+ def get_data(self):
3009+ ubuntu_pro_info = get_ubuntu_pro_info()
3010+
3011+ return json.dumps(ubuntu_pro_info, separators=(",", ":"))
3012+
3013+
3014+def get_ubuntu_pro_info():
3015+ try:
3016+ completed_process = subprocess.run(
3017+ ["ua", "status", "--format", "json"],
3018+ encoding="utf8", stdout=subprocess.PIPE)
3019+ except FileNotFoundError:
3020+ return {
3021+ "errors": [{
3022+ "message": "ubuntu-advantage-tools not found.",
3023+ "message_code": "tools-error",
3024+ "service": None,
3025+ "type": "system",
3026+ }],
3027+ "result": "failure",
3028+ }
3029+ else:
3030+ return json.loads(completed_process.stdout)
3031diff --git a/landscape/client/package/changer.py b/landscape/client/package/changer.py
3032index d4042ae..31253d2 100644
3033--- a/landscape/client/package/changer.py
3034+++ b/landscape/client/package/changer.py
3035@@ -1,5 +1,4 @@
3036 import logging
3037-import base64
3038 import time
3039 import os
3040 import pwd
3041@@ -14,6 +13,7 @@ from landscape.constants import (
3042 UNKNOWN_PACKAGE_DATA_TIMEOUT)
3043
3044 from landscape.lib.config import get_bindir
3045+from landscape.lib import base64
3046 from landscape.lib.fs import create_binary_file
3047 from landscape.lib.log import log_failure
3048 from landscape.client.package.reporter import find_reporter_command
3049@@ -173,7 +173,7 @@ class PackageChanger(PackageTaskHandler):
3050 hash_ids = {}
3051 for hash, id, deb in binaries:
3052 create_binary_file(os.path.join(binaries_path, "%d.deb" % id),
3053- base64.decodestring(deb))
3054+ base64.decodebytes(deb))
3055 hash_ids[hash] = id
3056 self._store.set_hash_ids(hash_ids)
3057 self._facade.add_channel_deb_dir(binaries_path)
3058diff --git a/landscape/client/package/reporter.py b/landscape/client/package/reporter.py
3059index 39b67df..0ac4381 100644
3060--- a/landscape/client/package/reporter.py
3061+++ b/landscape/client/package/reporter.py
3062@@ -3,6 +3,7 @@ try:
3063 except ImportError:
3064 import urllib.parse as urlparse
3065
3066+import locale
3067 import logging
3068 import time
3069 import os
3070@@ -866,6 +867,10 @@ class FakeReporter(PackageReporter):
3071
3072
3073 def main(args):
3074+ # Force UTF-8 encoding only for the reporter, thus allowing libapt-pkg to
3075+ # return unmangled descriptions.
3076+ locale.setlocale(locale.LC_CTYPE, ("C", "UTF-8"))
3077+
3078 if "FAKE_GLOBAL_PACKAGE_STORE" in os.environ:
3079 return run_task_handler(FakeGlobalReporter, args)
3080 elif "FAKE_PACKAGE_STORE" in os.environ:
3081diff --git a/landscape/client/package/tests/test_changer.py b/landscape/client/package/tests/test_changer.py
3082index db97d94..fe2c858 100644
3083--- a/landscape/client/package/tests/test_changer.py
3084+++ b/landscape/client/package/tests/test_changer.py
3085@@ -1,5 +1,4 @@
3086 # -*- encoding: utf-8 -*-
3087-import base64
3088 import time
3089 import sys
3090 import os
3091@@ -16,6 +15,7 @@ from landscape.lib.apt.package.facade import (
3092 from landscape.lib.apt.package.testing import (
3093 HASH1, HASH2, HASH3, PKGDEB1, PKGDEB2,
3094 AptFacadeHelper, SimpleRepositoryHelper)
3095+from landscape.lib import base64
3096 from landscape.lib.fs import create_text_file, read_text_file, touch_file
3097 from landscape.lib.testing import StubProcessFactory, FakeReactor
3098 from landscape.client.package.changer import (
3099@@ -849,9 +849,9 @@ class AptPackageChangerTest(LandscapeTest):
3100
3101 binaries_path = self.config.binaries_path
3102 self.assertFileContent(os.path.join(binaries_path, "111.deb"),
3103- base64.decodestring(PKGDEB1))
3104+ base64.decodebytes(PKGDEB1))
3105 self.assertFileContent(os.path.join(binaries_path, "222.deb"),
3106- base64.decodestring(PKGDEB2))
3107+ base64.decodebytes(PKGDEB2))
3108 self.assertEqual(
3109 self.facade.get_channels(),
3110 self.get_binaries_channels(binaries_path))
3111diff --git a/landscape/client/package/tests/test_releaseupgrader.py b/landscape/client/package/tests/test_releaseupgrader.py
3112index 13d03b3..9844a8a 100644
3113--- a/landscape/client/package/tests/test_releaseupgrader.py
3114+++ b/landscape/client/package/tests/test_releaseupgrader.py
3115@@ -433,12 +433,12 @@ class ReleaseUpgraderTest(LandscapeTest):
3116 upgrade_tool_filename = os.path.join(upgrade_tool_directory, "karmic")
3117 child_pid_filename = self.makeFile()
3118 fd = open(upgrade_tool_filename, "w")
3119- fd.write("#!/usr/bin/env python\n"
3120+ fd.write("#!/usr/bin/env python3\n"
3121 "import os\n"
3122 "import time\n"
3123 "import sys\n"
3124 "if __name__ == '__main__':\n"
3125- " print 'First parent'\n"
3126+ " print('First parent')\n"
3127 " pid = os.fork()\n"
3128 " if pid > 0:\n"
3129 " time.sleep(0.5)\n"
3130diff --git a/landscape/client/package/tests/test_reporter.py b/landscape/client/package/tests/test_reporter.py
3131index 6500aff..c4c4d0c 100644
3132--- a/landscape/client/package/tests/test_reporter.py
3133+++ b/landscape/client/package/tests/test_reporter.py
3134@@ -1,3 +1,4 @@
3135+import locale
3136 import sys
3137 import os
3138 import time
3139@@ -1231,6 +1232,31 @@ class PackageReporterAptTest(LandscapeTest):
3140 self.assertEqual("RESULT", main(["ARGS"]))
3141 m.assert_called_once_with(PackageReporter, ["ARGS"])
3142
3143+ def test_main_resets_locale(self):
3144+ """
3145+ Reporter entry point should reset encoding to utf-8, as libapt-pkg
3146+ encodes description with system encoding and python-apt decodes
3147+ them as utf-8 (LP: #1827857).
3148+ """
3149+ self._add_package_to_deb_dir(
3150+ self.repository_dir, "gosa", description=u"GOsa\u00B2")
3151+ self.facade.reload_channels()
3152+
3153+ # Set the only non-utf8 locale which we're sure exists.
3154+ # It behaves slightly differently than the bug, but fails on the
3155+ # same condition.
3156+ locale.setlocale(locale.LC_CTYPE, (None, None))
3157+ self.addCleanup(locale.resetlocale)
3158+
3159+ with mock.patch("landscape.client.package.reporter.run_task_handler"):
3160+ main([])
3161+
3162+ # With the actual package, the failure will occur looking up the
3163+ # description translation.
3164+ pkg = self.facade.get_packages_by_name("gosa")[0]
3165+ skel = self.facade.get_package_skeleton(pkg, with_info=True)
3166+ self.assertEqual(u"GOsa\u00B2", skel.description)
3167+
3168 def test_find_reporter_command_with_bindir(self):
3169 self.config.bindir = "/spam/eggs"
3170 command = find_reporter_command(self.config)
3171diff --git a/landscape/client/package/tests/test_taskhandler.py b/landscape/client/package/tests/test_taskhandler.py
3172index 2a37bd4..a6e998f 100644
3173--- a/landscape/client/package/tests/test_taskhandler.py
3174+++ b/landscape/client/package/tests/test_taskhandler.py
3175@@ -1,4 +1,5 @@
3176 import os
3177+from subprocess import CalledProcessError
3178
3179 from mock import patch, Mock, ANY
3180
3181@@ -103,7 +104,9 @@ class PackageTaskHandlerTest(LandscapeTest):
3182 self.handler.lsb_release_filename = self.makeFile()
3183
3184 # Go!
3185- result = self.handler.use_hash_id_db()
3186+ with patch("landscape.lib.lsb_release.check_output") as co_mock:
3187+ co_mock.side_effect = CalledProcessError(127, "")
3188+ result = self.handler.use_hash_id_db()
3189
3190 # The failure should be properly logged
3191 logging_mock.assert_called_with(
3192diff --git a/landscape/client/reactor.py b/landscape/client/reactor.py
3193index 501572e..df71d6c 100644
3194--- a/landscape/client/reactor.py
3195+++ b/landscape/client/reactor.py
3196@@ -2,6 +2,9 @@
3197 Extend the regular Twisted reactor with event-handling features.
3198 """
3199 from landscape.lib.reactor import EventHandlingReactor
3200+from landscape.client.lockfile import patch_lockfile
3201+
3202+patch_lockfile()
3203
3204
3205 class LandscapeReactor(EventHandlingReactor):
3206diff --git a/landscape/client/tests/test_amp.py b/landscape/client/tests/test_amp.py
3207index 04fb278..0960a38 100644
3208--- a/landscape/client/tests/test_amp.py
3209+++ b/landscape/client/tests/test_amp.py
3210@@ -1,9 +1,17 @@
3211-from twisted.internet.error import ConnectError
3212+import os
3213+import errno
3214+import subprocess
3215+import textwrap
3216+
3217+import mock
3218+
3219+from twisted.internet.error import ConnectError, CannotListenError
3220 from twisted.internet.task import Clock
3221
3222 from landscape.client.tests.helpers import LandscapeTest
3223 from landscape.client.deployment import Configuration
3224 from landscape.client.amp import ComponentPublisher, ComponentConnector, remote
3225+from landscape.client.reactor import LandscapeReactor
3226 from landscape.lib.amp import MethodCallError
3227 from landscape.lib.testing import FakeReactor
3228
3229@@ -159,3 +167,83 @@ class ComponentConnectorTest(LandscapeTest):
3230 effectively a no-op.
3231 """
3232 self.connector.disconnect()
3233+
3234+ @mock.patch("twisted.python.lockfile.kill")
3235+ def test_stale_locks_with_dead_pid(self, mock_kill):
3236+ """Publisher starts with stale lock."""
3237+ mock_kill.side_effect = [
3238+ OSError(errno.ESRCH, "No such process")]
3239+ sock_path = os.path.join(self.config.sockets_path, u"test.sock")
3240+ lock_path = u"{}.lock".format(sock_path)
3241+ # fake a PID which does not exist
3242+ os.symlink("-1", lock_path)
3243+
3244+ component = TestComponent()
3245+ # Test the actual Unix reactor implementation. Fakes won't do.
3246+ reactor = LandscapeReactor()
3247+ publisher = ComponentPublisher(component, reactor, self.config)
3248+
3249+ # Shouldn't raise the exception.
3250+ publisher.start()
3251+
3252+ # ensure stale lock was replaced
3253+ self.assertNotEqual("-1", os.readlink(lock_path))
3254+ mock_kill.assert_called_with(-1, 0)
3255+
3256+ publisher.stop()
3257+ reactor._cleanup()
3258+
3259+ @mock.patch("twisted.python.lockfile.kill")
3260+ def test_stale_locks_recycled_pid(self, mock_kill):
3261+ """Publisher starts with stale lock pointing to recycled process."""
3262+ mock_kill.side_effect = [
3263+ OSError(errno.EPERM, "Operation not permitted")]
3264+ sock_path = os.path.join(self.config.sockets_path, u"test.sock")
3265+ lock_path = u"{}.lock".format(sock_path)
3266+ # fake a PID recycled by a known process which isn't landscape (init)
3267+ os.symlink("1", lock_path)
3268+
3269+ component = TestComponent()
3270+ # Test the actual Unix reactor implementation. Fakes won't do.
3271+ reactor = LandscapeReactor()
3272+ publisher = ComponentPublisher(component, reactor, self.config)
3273+
3274+ # Shouldn't raise the exception.
3275+ publisher.start()
3276+
3277+ # ensure stale lock was replaced
3278+ self.assertNotEqual("1", os.readlink(lock_path))
3279+ mock_kill.assert_not_called()
3280+ self.assertFalse(publisher._port.lockFile.clean)
3281+
3282+ publisher.stop()
3283+ reactor._cleanup()
3284+
3285+ @mock.patch("twisted.python.lockfile.kill")
3286+ def test_with_valid_lock(self, mock_kill):
3287+ """Publisher raises lock error if a valid lock is held."""
3288+ sock_path = os.path.join(self.config.sockets_path, u"test.sock")
3289+ lock_path = u"{}.lock".format(sock_path)
3290+ # fake a landscape process
3291+ app = self.makeFile(textwrap.dedent("""\
3292+ #!/usr/bin/python3
3293+ import time
3294+ time.sleep(10)
3295+ """), basename="landscape-manager")
3296+ os.chmod(app, 0o755)
3297+ call = subprocess.Popen([app])
3298+ self.addCleanup(call.terminate)
3299+ os.symlink(str(call.pid), lock_path)
3300+
3301+ component = TestComponent()
3302+ # Test the actual Unix reactor implementation. Fakes won't do.
3303+ reactor = LandscapeReactor()
3304+ publisher = ComponentPublisher(component, reactor, self.config)
3305+
3306+ with self.assertRaises(CannotListenError):
3307+ publisher.start()
3308+
3309+ # ensure lock was not replaced
3310+ self.assertEqual(str(call.pid), os.readlink(lock_path))
3311+ mock_kill.assert_called_with(call.pid, 0)
3312+ reactor._cleanup()
3313diff --git a/landscape/client/tests/test_configuration.py b/landscape/client/tests/test_configuration.py
3314index 36d6d3e..d925d4e 100644
3315--- a/landscape/client/tests/test_configuration.py
3316+++ b/landscape/client/tests/test_configuration.py
3317@@ -4,6 +4,7 @@ import mock
3318 from functools import partial
3319 import os
3320 import sys
3321+import textwrap
3322 import unittest
3323
3324 from twisted.internet.defer import succeed, fail, Deferred
3325@@ -11,7 +12,7 @@ from twisted.python.compat import iteritems
3326
3327 from landscape.lib.compat import ConfigParser
3328 from landscape.lib.compat import StringIO
3329-from landscape.client.broker.registration import RegistrationError
3330+from landscape.client.broker.registration import RegistrationError, Identity
3331 from landscape.client.broker.tests.helpers import (
3332 RemoteBrokerHelper, BrokerConfigurationHelper)
3333 from landscape.client.configuration import (
3334@@ -21,7 +22,8 @@ from landscape.client.configuration import (
3335 ImportOptionError, store_public_key_data,
3336 bootstrap_tree, got_connection, success, failure, exchange_failure,
3337 handle_registration_errors, done, got_error, report_registration_outcome,
3338- determine_exit_code, is_registered)
3339+ determine_exit_code, is_registered, registration_info_text,
3340+ EXIT_NOT_REGISTERED)
3341 from landscape.lib.amp import MethodCallError
3342 from landscape.lib.fetch import HTTPCodeError, PyCurlError
3343 from landscape.lib.fs import read_binary_file
3344@@ -2233,3 +2235,72 @@ class IsRegisteredTest(LandscapeTest):
3345 self.persist.set("registration.secure-id", "super-secure")
3346 self.persist.save()
3347 self.assertTrue(is_registered(self.config))
3348+
3349+
3350+class RegistrationInfoTest(LandscapeTest):
3351+
3352+ helpers = [BrokerConfigurationHelper]
3353+
3354+ def setUp(self):
3355+ super(RegistrationInfoTest, self).setUp()
3356+ self.custom_args = ['hello.py'] # Fake python script name
3357+ self.account_name = 'world'
3358+ self.data_path = self.makeDir()
3359+ self.config_text = textwrap.dedent("""
3360+ [client]
3361+ computer_title = hello
3362+ account_name = {}
3363+ data_path = {}
3364+ """.format(self.account_name, self.data_path))
3365+
3366+ def test_not_registered(self):
3367+ '''False when client is not registered'''
3368+ config_filename = self.config.default_config_filenames[0]
3369+ self.makeFile(self.config_text, path=config_filename)
3370+ self.config.load(self.custom_args)
3371+ text = registration_info_text(self.config, False)
3372+ self.assertIn('False', text)
3373+ self.assertNotIn(self.account_name, text)
3374+
3375+ def test_registered(self):
3376+ '''
3377+ When client is registered, then the text should display as True and
3378+ account name should be present
3379+ '''
3380+ config_filename = self.config.default_config_filenames[0]
3381+ self.makeFile(self.config_text, path=config_filename)
3382+ self.config.load(self.custom_args)
3383+ text = registration_info_text(self.config, True)
3384+ self.assertIn('True', text)
3385+ self.assertIn(self.account_name, text)
3386+
3387+ def test_custom_config_path(self):
3388+ '''The custom config path should show up in the text'''
3389+ custom_path = self.makeFile(self.config_text)
3390+ self.custom_args += ['-c', custom_path]
3391+ self.config.load(self.custom_args)
3392+ text = registration_info_text(self.config, False)
3393+ self.assertIn(custom_path, text)
3394+
3395+ def test_data_path(self):
3396+ '''The config data path should show in the text'''
3397+ config_filename = self.config.default_config_filenames[0]
3398+ self.makeFile(self.config_text, path=config_filename)
3399+ self.config.load(self.custom_args)
3400+ text = registration_info_text(self.config, False)
3401+ self.assertIn(self.data_path, text)
3402+
3403+ def test_registered_exit_code(self):
3404+ '''Returns exit code zero when client is registered'''
3405+ Identity.secure_id = 'test' # Simulate successful registration
3406+ exception = self.assertRaises(
3407+ SystemExit, main, ["--is-registered", "--silent"],
3408+ print=noop_print)
3409+ self.assertEqual(0, exception.code)
3410+
3411+ def test_not_registered_exit_code(self):
3412+ '''Returns special return code when client is not registered'''
3413+ exception = self.assertRaises(
3414+ SystemExit, main, ["--is-registered", "--silent"],
3415+ print=noop_print)
3416+ self.assertEqual(EXIT_NOT_REGISTERED, exception.code)
3417diff --git a/landscape/client/tests/test_lockfile.py b/landscape/client/tests/test_lockfile.py
3418new file mode 100644
3419index 0000000..3b61c9f
3420--- /dev/null
3421+++ b/landscape/client/tests/test_lockfile.py
3422@@ -0,0 +1,21 @@
3423+import os
3424+import subprocess
3425+import textwrap
3426+
3427+from landscape.client import lockfile
3428+from landscape.client.tests.helpers import LandscapeTest
3429+
3430+
3431+class LockFileTest(LandscapeTest):
3432+
3433+ def test_read_process_name(self):
3434+ app = self.makeFile(textwrap.dedent("""\
3435+ #!/usr/bin/python3
3436+ import time
3437+ time.sleep(10)
3438+ """), basename="my_fancy_app")
3439+ os.chmod(app, 0o755)
3440+ call = subprocess.Popen([app])
3441+ self.addCleanup(call.terminate)
3442+ proc_name = lockfile.get_process_name(call.pid)
3443+ self.assertEqual("my_fancy_app", proc_name)
3444diff --git a/landscape/client/user/provider.py b/landscape/client/user/provider.py
3445index 61cd933..6abd5e9 100644
3446--- a/landscape/client/user/provider.py
3447+++ b/landscape/client/user/provider.py
3448@@ -4,7 +4,7 @@ import csv
3449 import logging
3450 import subprocess
3451
3452-from twisted.python.compat import _PY3
3453+from landscape.lib.compat import _PY3
3454
3455
3456 class UserManagementError(Exception):
3457diff --git a/landscape/lib/apt/package/facade.py b/landscape/lib/apt/package/facade.py
3458index 0c9114a..37e265e 100644
3459--- a/landscape/lib/apt/package/facade.py
3460+++ b/landscape/lib/apt/package/facade.py
3461@@ -164,9 +164,10 @@ class AptFacade(object):
3462 self._dpkg_status = os.path.join(dpkg_dir, "status")
3463 if not os.path.exists(self._dpkg_status):
3464 create_text_file(self._dpkg_status, "")
3465- # Apt will fail if it does not have a keyring. It does not care if
3466- # the keyring is empty.
3467- touch_file(os.path.join(apt_dir, "trusted.gpg"))
3468+ # Apt will fail if it does not have a keyring. It does not care if
3469+ # the keyring is empty. (Do not create one if dir exists LP: #1973202)
3470+ if not os.path.isdir(os.path.join(apt_dir, "trusted.gpg.d")):
3471+ touch_file(os.path.join(apt_dir, "trusted.gpg"))
3472
3473 def _ensure_sub_dir(self, sub_dir):
3474 """Ensure that a dir in the Apt root exists."""
3475@@ -605,9 +606,12 @@ class AptFacade(object):
3476 all_info = ["The following packages have unmet dependencies:"]
3477 for package in sorted(broken_packages, key=attrgetter("name")):
3478 found_dependency_error = False
3479+ # Fetch candidate version from our install list because
3480+ # apt-2.1.5 resets broken packages candidate.
3481+ candidate = next(v._cand for v in self._version_installs
3482+ if v.package == package)
3483 for dep_type in ["PreDepends", "Depends", "Conflicts", "Breaks"]:
3484- dependencies = package.candidate._cand.depends_list.get(
3485- dep_type, [])
3486+ dependencies = candidate.depends_list.get(dep_type, [])
3487 for dependency in dependencies:
3488 if self._is_dependency_satisfied(dependency, dep_type):
3489 continue
3490@@ -744,9 +748,18 @@ class AptFacade(object):
3491 # Set the candidate version, so that the version we want to
3492 # install actually is the one getting installed.
3493 version.package.candidate = version
3494+
3495+ # Flag the package as manual if it's a new install, otherwise
3496+ # preserve the auto flag. This should preserve explicitly
3497+ # installed packages from auto-removal, while allowing upgrades
3498+ # of auto-removable packages.
3499+ is_manual = (
3500+ not version.package.installed or
3501+ not version.package.is_auto_installed)
3502+
3503 # Set auto_fix=False to avoid removing the package we asked to
3504 # install when we need to resolve dependencies.
3505- version.package.mark_install(auto_fix=False)
3506+ version.package.mark_install(auto_fix=False, from_user=is_manual)
3507 self._package_installs.add(version.package)
3508 fixer.clear(version.package._pkg)
3509 fixer.protect(version.package._pkg)
3510diff --git a/landscape/lib/apt/package/skeleton.py b/landscape/lib/apt/package/skeleton.py
3511index eb9ba25..846ab58 100644
3512--- a/landscape/lib/apt/package/skeleton.py
3513+++ b/landscape/lib/apt/package/skeleton.py
3514@@ -1,9 +1,8 @@
3515+from landscape.lib.compat import unicode, _PY3
3516 from landscape.lib.hashlib import sha1
3517
3518 import apt_pkg
3519
3520-from twisted.python.compat import unicode, _PY3
3521-
3522
3523 PACKAGE = 1 << 0
3524 PROVIDES = 1 << 1
3525diff --git a/landscape/lib/apt/package/testing.py b/landscape/lib/apt/package/testing.py
3526index c3e3213..4e8b551 100644
3527--- a/landscape/lib/apt/package/testing.py
3528+++ b/landscape/lib/apt/package/testing.py
3529@@ -1,4 +1,3 @@
3530-import base64
3531 import os
3532 import time
3533
3534@@ -8,6 +7,7 @@ import apt_pkg
3535 from landscape.lib.apt.package.facade import AptFacade
3536 from landscape.lib.fs import append_binary_file
3537 from landscape.lib.fs import create_binary_file
3538+from landscape.lib import base64
3539
3540
3541 class AptFacadeHelper(object):
3542@@ -322,9 +322,9 @@ PKGDEB_OR_RELATIONS = (
3543 ).encode("ascii")
3544
3545
3546-HASH1 = base64.decodestring(b"/ezv4AefpJJ8DuYFSq4RiEHJYP4=")
3547-HASH2 = base64.decodestring(b"glP4DwWOfMULm0AkRXYsH/exehc=")
3548-HASH3 = base64.decodestring(b"NJM05mj86veaSInYxxqL1wahods=")
3549+HASH1 = base64.decodebytes(b"/ezv4AefpJJ8DuYFSq4RiEHJYP4=")
3550+HASH2 = base64.decodebytes(b"glP4DwWOfMULm0AkRXYsH/exehc=")
3551+HASH3 = base64.decodebytes(b"NJM05mj86veaSInYxxqL1wahods=")
3552 HASH_MINIMAL = b"6\xce\x8f\x1bM\x82MWZ\x1a\xffjAc(\xdb(\xa1\x0eG"
3553 HASH_SIMPLE_RELATIONS = (
3554 b"'#\xab&k\xe6\xf5E\xcfB\x9b\xceO7\xe6\xec\xa9\xddY\xaa")
3555@@ -339,7 +339,7 @@ HASH_OR_RELATIONS = (
3556 def create_deb(target_dir, pkg_name, pkg_data):
3557 """Create a Debian package in the specified C{target_dir}."""
3558 path = os.path.join(target_dir, pkg_name)
3559- data = base64.decodestring(pkg_data)
3560+ data = base64.decodebytes(pkg_data)
3561 create_binary_file(path, data)
3562
3563
3564diff --git a/landscape/lib/apt/package/tests/test_facade.py b/landscape/lib/apt/package/tests/test_facade.py
3565index 820f6ec..9ea0654 100644
3566--- a/landscape/lib/apt/package/tests/test_facade.py
3567+++ b/landscape/lib/apt/package/tests/test_facade.py
3568@@ -1812,6 +1812,7 @@ class AptFacadeTest(testing.HelperTestCase, testing.FSTestCase,
3569 self._add_system_package("foo")
3570 self.facade.reload_channels()
3571 [foo] = self.facade.get_packages_by_name("foo")
3572+ self.facade._version_installs.append(foo)
3573 self.facade._get_broken_packages = lambda: set([foo.package])
3574 self.assertEqual(
3575 ["The following packages have unmet dependencies:",
3576@@ -2601,6 +2602,72 @@ class AptFacadeTest(testing.HelperTestCase, testing.FSTestCase,
3577 [bar] = self.facade._cache.get_changes()
3578 self.assertTrue(bar.marked_upgrade)
3579
3580+ def test_changer_upgrade_keeps_auto(self):
3581+ """
3582+ An upgrade request should preserve an existing auto flag on the
3583+ upgraded package.
3584+ """
3585+ self._add_system_package(
3586+ "foo", control_fields={"Depends": "bar"})
3587+ self._add_system_package("bar", version="1.0")
3588+ deb_dir = self.makeDir()
3589+ self._add_package_to_deb_dir(deb_dir, "bar", version="2.0")
3590+ self.facade.add_channel_apt_deb(
3591+ "file://%s" % deb_dir, "./", trusted=True)
3592+ self.facade.reload_channels()
3593+ bar_1, bar_2 = sorted(self.facade.get_packages_by_name("bar"))
3594+ bar_1.package.mark_auto()
3595+
3596+ self.facade.mark_install(bar_2)
3597+ self.facade.mark_remove(bar_1)
3598+ self.patch_cache_commit()
3599+ self.facade.perform_changes()
3600+ [bar] = self.facade._cache.get_changes()
3601+ self.assertTrue(bar.marked_upgrade)
3602+ self.assertTrue(bar.is_auto_installed)
3603+
3604+ def test_changer_upgrade_keeps_manual(self):
3605+ """
3606+ An upgrade request should mark a package as manual if the installed
3607+ version is manual.
3608+ """
3609+ self._add_system_package(
3610+ "foo", control_fields={"Depends": "bar"})
3611+ self._add_system_package("bar", version="1.0")
3612+ deb_dir = self.makeDir()
3613+ self._add_package_to_deb_dir(deb_dir, "bar", version="2.0")
3614+ self.facade.add_channel_apt_deb(
3615+ "file://%s" % deb_dir, "./", trusted=True)
3616+ self.facade.reload_channels()
3617+ bar_1, bar_2 = sorted(self.facade.get_packages_by_name("bar"))
3618+
3619+ self.facade.mark_install(bar_2)
3620+ self.facade.mark_remove(bar_1)
3621+ self.patch_cache_commit()
3622+ self.facade.perform_changes()
3623+ [bar] = self.facade._cache.get_changes()
3624+ self.assertTrue(bar.marked_upgrade)
3625+ self.assertFalse(bar.is_auto_installed)
3626+
3627+ def test_changer_install_sets_manual(self):
3628+ """
3629+ An installation request should mark the new package as manually
3630+ installed.
3631+ """
3632+ deb_dir = self.makeDir()
3633+ self._add_package_to_deb_dir(deb_dir, "bar", version="2.0")
3634+ self.facade.add_channel_apt_deb(
3635+ "file://%s" % deb_dir, "./", trusted=True)
3636+ self.facade.reload_channels()
3637+ bar_2, = self.facade.get_packages_by_name("bar")
3638+
3639+ self.facade.mark_install(bar_2)
3640+ self.patch_cache_commit()
3641+ self.facade.perform_changes()
3642+ [bar] = self.facade._cache.get_changes()
3643+ self.assertTrue(bar.marked_upgrade)
3644+ self.assertFalse(bar.is_auto_installed)
3645+
3646 def test_mark_global_upgrade_held_packages(self):
3647 """
3648 If a package that is on hold is marked for upgrade,
3649diff --git a/landscape/lib/apt/package/tests/test_skeleton.py b/landscape/lib/apt/package/tests/test_skeleton.py
3650index 63737b5..a240d10 100644
3651--- a/landscape/lib/apt/package/tests/test_skeleton.py
3652+++ b/landscape/lib/apt/package/tests/test_skeleton.py
3653@@ -1,3 +1,4 @@
3654+import locale
3655 import unittest
3656
3657 from landscape.lib import testing
3658@@ -147,15 +148,22 @@ class SkeletonAptTest(BaseTestCase):
3659 def test_build_skeleton_with_unicode_and_non_ascii(self):
3660 """
3661 If with_unicode and with_info are passed to build_skeleton_apt,
3662- the description is decoded and non-ascii chars replaced.
3663+ the description is decoded.
3664 """
3665+ # Py2 used to convert to lossy ascii (thus LC_ in Makefile)
3666+ # Py3 doesn't, and python3-apt assumes UTF8 (LP: #1827857).
3667+ # If you revisit this test, also check reporter.main(), which
3668+ # should set this globally to the reporter process.
3669+ locale.setlocale(locale.LC_CTYPE, "C.UTF-8")
3670+ self.addCleanup(locale.resetlocale)
3671+
3672 self._add_package_to_deb_dir(
3673 self.skeleton_repository_dir, "pkg", description=u"T\xe9st")
3674 self.facade._cache.update(None)
3675 self.facade._cache.open(None)
3676 pkg = self.get_package("pkg")
3677 skeleton = build_skeleton_apt(pkg, with_unicode=True, with_info=True)
3678- self.assertEqual(u"T?st", skeleton.description)
3679+ self.assertEqual(u"T\u00E9st", skeleton.description)
3680
3681 def test_build_skeleton_minimal(self):
3682 """
3683diff --git a/landscape/lib/backoff.py b/landscape/lib/backoff.py
3684new file mode 100644
3685index 0000000..b2c7ac3
3686--- /dev/null
3687+++ b/landscape/lib/backoff.py
3688@@ -0,0 +1,53 @@
3689+import random
3690+
3691+
3692+class ExponentialBackoff:
3693+ """
3694+ Keeps track of a backoff delay that staggers down and staggers up
3695+ exponentially.
3696+ """
3697+
3698+ def __init__(self, start_delay, max_delay):
3699+
3700+ self._error_count = 0 # A tally of server errors
3701+
3702+ self._start_delay = start_delay
3703+ self._max_delay = max_delay
3704+
3705+ def decrease(self):
3706+ """Decreases error count with zero being the lowest"""
3707+ self._error_count -= 1
3708+ self._error_count = max(self._error_count, 0)
3709+
3710+ def increase(self):
3711+ """Increases error count but not higher than gives the max delay"""
3712+ if self.get_delay() < self._max_delay:
3713+ self._error_count += 1
3714+
3715+ def get_delay(self):
3716+ """
3717+ Calculates the delay using formula that gives this chart. In this
3718+ specific example start is 5 seconds and max is 60 seconds
3719+ Count Delay
3720+ 0 0
3721+ 1 5
3722+ 2 10
3723+ 3 20
3724+ 4 40
3725+ 5 60 (max)
3726+ """
3727+ if self._error_count:
3728+ delay = (2 ** (self._error_count - 1)) * self._start_delay
3729+ else:
3730+ delay = 0
3731+ return min(int(delay), self._max_delay)
3732+
3733+ def get_random_delay(self, stagger_fraction=0.25):
3734+ """
3735+ Adds randomness to the specified stagger of the delay. For example
3736+ for a delay of 12 and 25% stagger, it works out to 9 + rand(0,3)
3737+ """
3738+ delay = self.get_delay()
3739+ non_random_part = delay * (1-stagger_fraction)
3740+ random_part = delay * stagger_fraction * random.random()
3741+ return int(non_random_part + random_part)
3742diff --git a/landscape/lib/base64.py b/landscape/lib/base64.py
3743new file mode 100644
3744index 0000000..148a2d5
3745--- /dev/null
3746+++ b/landscape/lib/base64.py
3747@@ -0,0 +1,8 @@
3748+from __future__ import absolute_import
3749+
3750+from landscape.lib.compat import _PY3
3751+
3752+if _PY3:
3753+ from base64 import decodebytes # noqa
3754+else:
3755+ from base64 import decodestring as decodebytes # noqa
3756diff --git a/landscape/lib/bpickle.py b/landscape/lib/bpickle.py
3757index 93d5760..4f0d5ca 100644
3758--- a/landscape/lib/bpickle.py
3759+++ b/landscape/lib/bpickle.py
3760@@ -32,7 +32,7 @@ This file is modified from the original to work with python3, but should be
3761 wire compatible and behave the same way (bugs notwithstanding).
3762 """
3763
3764-from twisted.python.compat import _PY3
3765+from landscape.lib.compat import long, _PY3
3766
3767 dumps_table = {}
3768 loads_table = {}
3769diff --git a/landscape/lib/compat.py b/landscape/lib/compat.py
3770index 115f86a..2452524 100644
3771--- a/landscape/lib/compat.py
3772+++ b/landscape/lib/compat.py
3773@@ -1,6 +1,6 @@
3774 # flake8: noqa
3775
3776-from twisted.python.compat import _PY3
3777+_PY3 = str != bytes
3778
3779
3780 if _PY3:
3781@@ -13,6 +13,8 @@ if _PY3:
3782 from io import StringIO
3783 stringio = cstringio = StringIO
3784 from builtins import input
3785+ unicode = str
3786+ long = int
3787
3788 else:
3789 import cPickle
3790@@ -24,3 +26,5 @@ else:
3791 stringio = StringIO
3792 from cStringIO import StringIO as cstringio
3793 input = raw_input
3794+ long = long
3795+ unicode = unicode
3796diff --git a/landscape/lib/config.py b/landscape/lib/config.py
3797index 7893649..5880876 100644
3798--- a/landscape/lib/config.py
3799+++ b/landscape/lib/config.py
3800@@ -228,7 +228,7 @@ class BaseConfiguration(object):
3801 raise_errors=False, write_empty_values=True)
3802 except ConfigObjError as e:
3803 logger = getLogger()
3804- logger.warn(str(e))
3805+ logger.warn("ERROR at {}: {}".format(config_source, str(e)))
3806 # Good configuration values are recovered here
3807 config_obj = e.config
3808 return config_obj
3809diff --git a/landscape/lib/disk.py b/landscape/lib/disk.py
3810index 7bf5884..2a8ad35 100644
3811--- a/landscape/lib/disk.py
3812+++ b/landscape/lib/disk.py
3813@@ -4,13 +4,13 @@ import os
3814 import re
3815 import codecs
3816
3817-from twisted.python.compat import _PY3
3818+from landscape.lib.compat import _PY3
3819
3820
3821 # List of filesystem types authorized when generating disk use statistics.
3822 STABLE_FILESYSTEMS = frozenset(
3823 ["ext", "ext2", "ext3", "ext4", "reiserfs", "ntfs", "msdos", "dos", "vfat",
3824- "xfs", "hpfs", "jfs", "ufs", "hfs", "hfsplus", "simfs"])
3825+ "xfs", "hpfs", "jfs", "ufs", "hfs", "hfsplus", "simfs", "drvfs", "lxfs"])
3826
3827
3828 EXTRACT_DEVICE = re.compile("([a-z]+)[0-9]*")
3829diff --git a/landscape/lib/gpg.py b/landscape/lib/gpg.py
3830index 66400d1..58218c5 100644
3831--- a/landscape/lib/gpg.py
3832+++ b/landscape/lib/gpg.py
3833@@ -1,6 +1,9 @@
3834+import itertools
3835 import shutil
3836 import tempfile
3837
3838+from glob import glob
3839+
3840 from twisted.internet.utils import getProcessOutputAndValue
3841
3842
3843@@ -8,14 +11,15 @@ class InvalidGPGSignature(Exception):
3844 """Raised when the gpg signature for a given file is invalid."""
3845
3846
3847-def gpg_verify(filename, signature, gpg="/usr/bin/gpg"):
3848+def gpg_verify(filename, signature, gpg="/usr/bin/gpg", apt_dir="/etc/apt"):
3849 """Verify the GPG signature of a file.
3850
3851 @param filename: Path to the file to verify the signature against.
3852 @param signature: Path to signature to use.
3853 @param gpg: Optionally, path to the GPG binary to use.
3854+ @param apt_dir: Optionally, path to apt trusted keyring.
3855 @return: a C{Deferred} resulting in C{True} if the signature is
3856- valid, C{False} otherwise.
3857+ valid, C{False} otherwise.
3858 """
3859
3860 def remove_gpg_home(ignored):
3861@@ -32,9 +36,17 @@ def gpg_verify(filename, signature, gpg="/usr/bin/gpg"):
3862 "code='%d')" % (gpg, out, err, code))
3863
3864 gpg_home = tempfile.mkdtemp()
3865- args = ("--no-options", "--homedir", gpg_home, "--no-default-keyring",
3866- "--ignore-time-conflict", "--keyring", "/etc/apt/trusted.gpg",
3867- "--verify", signature, filename)
3868+ keyrings = tuple(itertools.chain(*[
3869+ ("--keyring", keyring)
3870+ for keyring in sorted(
3871+ glob("{}/trusted.gpg".format(apt_dir)) +
3872+ glob("{}/trusted.gpg.d/*.gpg".format(apt_dir))
3873+ )
3874+ ]))
3875+ args = (
3876+ "--no-options", "--homedir", gpg_home, "--no-default-keyring",
3877+ "--ignore-time-conflict"
3878+ ) + keyrings + ("--verify", signature, filename)
3879
3880 result = getProcessOutputAndValue(gpg, args=args)
3881 result.addBoth(remove_gpg_home)
3882diff --git a/landscape/lib/lsb_release.py b/landscape/lib/lsb_release.py
3883index 6e05fd8..cfc3cd3 100644
3884--- a/landscape/lib/lsb_release.py
3885+++ b/landscape/lib/lsb_release.py
3886@@ -1,26 +1,59 @@
3887-"""Get information from /etc/lsb_release."""
3888+"""Get information from /usr/bin/lsb_release."""
3889+import os
3890+from subprocess import CalledProcessError, check_output
3891
3892-LSB_RELEASE_FILENAME = "/etc/lsb-release"
3893-LSB_RELEASE_INFO_KEYS = {"DISTRIB_ID": "distributor-id",
3894- "DISTRIB_DESCRIPTION": "description",
3895- "DISTRIB_RELEASE": "release",
3896- "DISTRIB_CODENAME": "code-name"}
3897+LSB_RELEASE = "/usr/bin/lsb_release"
3898+LSB_RELEASE_FILENAME = "/etc/lsb_release"
3899+LSB_RELEASE_FILE_KEYS = {
3900+ "DISTRIB_ID": "distributor-id",
3901+ "DISTRIB_DESCRIPTION": "description",
3902+ "DISTRIB_RELEASE": "release",
3903+ "DISTRIB_CODENAME": "code-name",
3904+}
3905
3906
3907-def parse_lsb_release(lsb_release_filename):
3908- """Return a C{dict} holding information about the system LSB release.
3909+def parse_lsb_release(lsb_release_filename=None):
3910+ """
3911+ Returns a C{dict} holding information about the system LSB release.
3912+ Reads from C{lsb_release_filename} if it exists, else calls
3913+ C{LSB_RELEASE}
3914+ """
3915+ if lsb_release_filename and os.path.exists(lsb_release_filename):
3916+ return parse_lsb_release_file(lsb_release_filename)
3917+
3918+ with open(os.devnull, 'w') as FNULL:
3919+ try:
3920+ lsb_info = check_output([LSB_RELEASE, "-as"], stderr=FNULL)
3921+ except (CalledProcessError, FileNotFoundError):
3922+ # Fall back to reading file, even if it doesn't exist.
3923+ return parse_lsb_release_file(lsb_release_filename)
3924+ else:
3925+ dist, desc, release, code_name, _ = lsb_info.decode().split("\n")
3926+
3927+ return {
3928+ "distributor-id": dist,
3929+ "release": release,
3930+ "code-name": code_name,
3931+ "description": desc,
3932+ }
3933+
3934
3935- @raises: An IOError exception if C{lsb_release_filename} could not be read.
3936+def parse_lsb_release_file(filename):
3937+ """
3938+ Returns a C{dict} holding information about the system LSB release
3939+ by attempting to parse C{filename}.
3940+
3941+ @raises: A FileNotFoundError if C{filename} does not exist.
3942 """
3943- fd = open(lsb_release_filename, "r")
3944 info = {}
3945- try:
3946+
3947+ with open(filename) as fd:
3948 for line in fd:
3949 key, value = line.split("=")
3950- if key in LSB_RELEASE_INFO_KEYS:
3951- key = LSB_RELEASE_INFO_KEYS[key.strip()]
3952+
3953+ if key in LSB_RELEASE_FILE_KEYS:
3954+ key = LSB_RELEASE_FILE_KEYS[key.strip()]
3955 value = value.strip().strip('"')
3956 info[key] = value
3957- finally:
3958- fd.close()
3959+
3960 return info
3961diff --git a/landscape/lib/message.py b/landscape/lib/message.py
3962index ed742af..7ed55da 100644
3963--- a/landscape/lib/message.py
3964+++ b/landscape/lib/message.py
3965@@ -1,6 +1,6 @@
3966 """Helpers for reliable persistent message queues."""
3967
3968-ANCIENT = 1
3969+RESYNC = object() # Used as a flag indicating a resync is needed
3970
3971
3972 def got_next_expected(store, next_expected):
3973@@ -10,6 +10,12 @@ def got_next_expected(store, next_expected):
3974 wants next; this will do various things based on what *this* side
3975 has in its outbound queue store.
3976
3977+ 0. The peer expects a sequence number from the server that is too high, the
3978+ difference greater than pending messages in the peer. We flush the older
3979+ messages in queue b/c the server does not want old or ancient messages,
3980+ however we also reset the offset so that the pending messages are
3981+ resent. Then we resynchronize by returning RESYNC. See LP: #1917540
3982+
3983 1. The peer expects a sequence greater than what we last
3984 sent. This is the common case and generally it should be
3985 expecting last_sent_sequence+len(messages_sent)+1.
3986@@ -25,18 +31,22 @@ def got_next_expected(store, next_expected):
3987 from that message.
3988
3989 If the next expected sequence from the server refers to a message
3990- older than we have, then L{ANCIENT} will be returned.
3991+ older than we have, then L{RESYNC} will be returned.
3992 """
3993 ret = None
3994 old_sequence = store.get_sequence()
3995- if next_expected > old_sequence:
3996+ if (next_expected - old_sequence) > store.count_pending_messages():
3997+ store.delete_old_messages() # Flush queue from previous iteration
3998+ pending_offset = 0 # This means current messages will be resent
3999+ ret = RESYNC
4000+ elif next_expected > old_sequence:
4001 store.delete_old_messages()
4002 pending_offset = next_expected - old_sequence
4003 elif next_expected < (old_sequence - store.get_pending_offset()):
4004 # "Ancient": The other side wants messages we don't have,
4005 # so let's just reset our counter to what it expects.
4006 pending_offset = 0
4007- ret = ANCIENT
4008+ ret = RESYNC
4009 else:
4010 # No messages transferred, or
4011 # "Old": We'll try to send these old messages that the
4012diff --git a/landscape/lib/network.py b/landscape/lib/network.py
4013index 0ee336a..c9801c2 100644
4014--- a/landscape/lib/network.py
4015+++ b/landscape/lib/network.py
4016@@ -11,7 +11,7 @@ import errno
4017 import logging
4018
4019 import netifaces
4020-from twisted.python.compat import long
4021+from landscape.lib.compat import long, _PY3
4022
4023 __all__ = ["get_active_device_info", "get_network_traffic"]
4024
4025@@ -35,32 +35,14 @@ def is_up(flags):
4026 return flags & 1
4027
4028
4029-def get_active_interfaces():
4030- """Generator yields (active network interface name, address data) tuples.
4031+def is_active(ifaddresses):
4032+ """Checks if interface address data has an IP address
4033
4034- Address data is formatted exactly like L{netifaces.ifaddresses}, e.g.::
4035-
4036- ('eth0', {
4037- AF_LINK: [
4038- {'addr': '...', 'broadcast': '...'}, ],
4039- AF_INET: [
4040- {'addr': '...', 'broadcast': '...', 'netmask': '...'},
4041- {'addr': '...', 'broadcast': '...', 'netmask': '...'},
4042- ...],
4043- AF_INET6: [
4044- {'addr': '...', 'netmask': '...'},
4045- {'addr': '...', 'netmask': '...'},
4046- ...], })
4047-
4048- Interfaces with no IP address are ignored.
4049+ @param ifaddresses: a dict as returned by L{netifaces.ifaddresses}
4050 """
4051- for interface in netifaces.interfaces():
4052- ifaddresses = netifaces.ifaddresses(interface)
4053- # Skip interfaces with no IPv4 or IPv6 addresses.
4054- inet_addr = ifaddresses.get(netifaces.AF_INET, [{}])[0].get('addr')
4055- inet6_addr = ifaddresses.get(netifaces.AF_INET6, [{}])[0].get('addr')
4056- if inet_addr or inet6_addr:
4057- yield interface, ifaddresses
4058+ inet_addr = ifaddresses.get(netifaces.AF_INET, [{}])[0].get('addr')
4059+ inet6_addr = ifaddresses.get(netifaces.AF_INET6, [{}])[0].get('addr')
4060+ return bool(inet_addr or inet6_addr)
4061
4062
4063 def get_ip_addresses(ifaddresses):
4064@@ -69,8 +51,7 @@ def get_ip_addresses(ifaddresses):
4065 Returns the same structure as L{ifaddresses}, but filtered to keep
4066 IP addresses only.
4067
4068- @param ifaddresses: a dict as returned by L{netifaces.ifaddresses} or
4069- the address data in L{get_active_interfaces}'s output.
4070+ @param ifaddresses: a dict as returned by L{netifaces.ifaddresses}
4071 """
4072 results = {}
4073 if netifaces.AF_INET in ifaddresses:
4074@@ -88,8 +69,7 @@ def get_ip_addresses(ifaddresses):
4075 def get_broadcast_address(ifaddresses):
4076 """Return the broadcast address associated to an interface.
4077
4078- @param ifaddresses: a dict as returned by L{netifaces.ifaddresses} or
4079- the address data in L{get_active_interfaces}'s output.
4080+ @param ifaddresses: a dict as returned by L{netifaces.ifaddresses}
4081 """
4082 return ifaddresses[netifaces.AF_INET][0].get('broadcast', '0.0.0.0')
4083
4084@@ -97,8 +77,7 @@ def get_broadcast_address(ifaddresses):
4085 def get_netmask(ifaddresses):
4086 """Return the network mask associated to an interface.
4087
4088- @param ifaddresses: a dict as returned by L{netifaces.ifaddresses} or
4089- the address data in L{get_active_interfaces}'s output.
4090+ @param ifaddresses: a dict as returned by L{netifaces.ifaddresses}
4091 """
4092 return ifaddresses[netifaces.AF_INET][0].get('netmask', '')
4093
4094@@ -106,8 +85,7 @@ def get_netmask(ifaddresses):
4095 def get_ip_address(ifaddresses):
4096 """Return the first IPv4 address associated to the interface.
4097
4098- @param ifaddresses: a dict as returned by L{netifaces.ifaddresses} or
4099- the address data in L{get_active_interfaces}'s output.
4100+ @param ifaddresses: a dict as returned by L{netifaces.ifaddresses}
4101 """
4102 return ifaddresses[netifaces.AF_INET][0]['addr']
4103
4104@@ -118,8 +96,7 @@ def get_mac_address(ifaddresses):
4105 ie. six colon separated groups of two hexadecimal digits, if available;
4106 otherwise an empty string.
4107
4108- @param ifaddresses: a dict as returned by L{netifaces.ifaddresses} or
4109- the address data in L{get_active_interfaces}'s output.
4110+ @param ifaddresses: a dict as returned by L{netifaces.ifaddresses}
4111 """
4112 if netifaces.AF_LINK in ifaddresses:
4113 return ifaddresses[netifaces.AF_LINK][0].get('addr', '')
4114@@ -138,51 +115,104 @@ def get_flags(sock, interface):
4115 return struct.unpack("H", data[16:18])[0]
4116
4117
4118-def get_active_device_info(skipped_interfaces=("lo",),
4119- skip_vlan=True, skip_alias=True, extended=False):
4120+def get_default_interfaces():
4121+ """
4122+ Returns a list of interfaces with default routes
4123+ """
4124+ default_table = netifaces.gateways()['default']
4125+ interfaces = [gateway[1] for gateway in default_table.values()]
4126+ return interfaces
4127+
4128+
4129+def get_filtered_if_info(filters=(), extended=False):
4130 """
4131- Returns a dictionary containing information on each active network
4132- interface present on a machine.
4133+ Returns a dictionary containing info on each active network
4134+ interface that passes all `filters`.
4135+
4136+ A filter is a callable that returns True if the interface should be
4137+ skipped.
4138 """
4139 results = []
4140+
4141 try:
4142 sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM,
4143 socket.IPPROTO_IP)
4144- for interface, ifaddresses in get_active_interfaces():
4145- if interface in skipped_interfaces:
4146- continue
4147- if skip_vlan and "." in interface:
4148+
4149+ for interface in netifaces.interfaces():
4150+ if any(f(interface) for f in filters):
4151 continue
4152- if skip_alias and ":" in interface:
4153+
4154+ ifaddresses = netifaces.ifaddresses(interface)
4155+ if not is_active(ifaddresses):
4156 continue
4157- flags = get_flags(sock, interface.encode())
4158+
4159+ ifencoded = interface.encode()
4160+ flags = get_flags(sock, ifencoded)
4161 if not is_up(flags):
4162 continue
4163- interface_info = {"interface": interface}
4164- interface_info["flags"] = flags
4165- speed, duplex = get_network_interface_speed(
4166- sock, interface.encode())
4167- interface_info["speed"] = speed
4168- interface_info["duplex"] = duplex
4169+
4170 ip_addresses = get_ip_addresses(ifaddresses)
4171+ if not extended and netifaces.AF_INET not in ip_addresses:
4172+ # Skip interfaces with no IPv4 addr unless extended to
4173+ # keep backwards compatibility with single-IPv4 addr
4174+ # support.
4175+ continue
4176+
4177+ ifinfo = {"interface": interface}
4178+ ifinfo["flags"] = flags
4179+ ifinfo["speed"], ifinfo["duplex"] = get_network_interface_speed(
4180+ sock, ifencoded)
4181+
4182 if extended:
4183- interface_info["ip_addresses"] = ip_addresses
4184+ ifinfo["ip_addresses"] = ip_addresses
4185+
4186 if netifaces.AF_INET in ip_addresses:
4187- interface_info["ip_address"] = get_ip_address(ifaddresses)
4188- interface_info["mac_address"] = get_mac_address(ifaddresses)
4189- interface_info["broadcast_address"] = get_broadcast_address(
4190+ ifinfo["ip_address"] = get_ip_address(ifaddresses)
4191+ ifinfo["mac_address"] = get_mac_address(ifaddresses)
4192+ ifinfo["broadcast_address"] = get_broadcast_address(
4193 ifaddresses)
4194- interface_info["netmask"] = get_netmask(ifaddresses)
4195- # Skip interfaces with no IPv4 address in non-extended mode
4196- # to keep backwards compatibility with single-IPv4 addr support.
4197- if netifaces.AF_INET in ip_addresses or extended:
4198- results.append(interface_info)
4199+ ifinfo["netmask"] = get_netmask(ifaddresses)
4200+
4201+ results.append(ifinfo)
4202 finally:
4203- del sock
4204+ sock.close()
4205
4206 return results
4207
4208
4209+def get_active_device_info(skipped_interfaces=("lo",),
4210+ skip_vlan=True, skip_alias=True,
4211+ extended=False, default_only=False):
4212+ def filter_local(interface):
4213+ return interface in skipped_interfaces
4214+
4215+ def filter_vlan(interface):
4216+ return "." in interface
4217+
4218+ def filter_alias(interface):
4219+ return ":" in interface
4220+
4221+ # Get default interfaces here because it could be expensive and
4222+ # there's no reason to do it more than once.
4223+ default_ifs = get_default_interfaces()
4224+
4225+ def filter_default(interface):
4226+ return default_only and interface not in default_ifs
4227+
4228+ # Tap interfaces can be extremely numerous, slowing us down
4229+ # significantly.
4230+ def filter_tap(interface):
4231+ return interface.startswith("tap")
4232+
4233+ return get_filtered_if_info(filters=(
4234+ filter_tap,
4235+ filter_local,
4236+ filter_vlan,
4237+ filter_alias,
4238+ filter_default,
4239+ ), extended=extended)
4240+
4241+
4242 def get_network_traffic(source_file="/proc/net/dev"):
4243 """
4244 Retrieves an array of information regarding the network activity per
4245@@ -246,13 +276,16 @@ def get_network_interface_speed(sock, interface_name):
4246 speed = -1
4247 try:
4248 fcntl.ioctl(sock, SIOCETHTOOL, packed) # Status ioctl() call
4249- res = status_cmd.tostring()
4250+ if _PY3:
4251+ res = status_cmd.tobytes()
4252+ else:
4253+ res = status_cmd.tostring()
4254 speed, duplex = struct.unpack("12xHB28x", res)
4255- except IOError as e:
4256+ except (IOError, OSError) as e:
4257 if e.errno == errno.EPERM:
4258- logging.warn("Could not determine network interface speed, "
4259- "operation not permitted.")
4260- elif e.errno != errno.EOPNOTSUPP:
4261+ logging.warning("Could not determine network interface speed, "
4262+ "operation not permitted.")
4263+ elif e.errno != errno.EOPNOTSUPP and e.errno != errno.EINVAL:
4264 raise e
4265 speed = -1
4266 duplex = False
4267diff --git a/landscape/lib/schema.py b/landscape/lib/schema.py
4268index e2d65d0..f9397db 100644
4269--- a/landscape/lib/schema.py
4270+++ b/landscape/lib/schema.py
4271@@ -13,6 +13,12 @@ class Constant(object):
4272 self.value = value
4273
4274 def coerce(self, value):
4275+ if isinstance(self.value, str) and isinstance(value, bytes):
4276+ try:
4277+ value = value.decode()
4278+ except UnicodeDecodeError:
4279+ pass
4280+
4281 if value != self.value:
4282 raise InvalidError("%r != %r" % (value, self.value))
4283 return value
4284@@ -65,11 +71,19 @@ class Float(object):
4285
4286
4287 class Bytes(object):
4288- """A binary string."""
4289+ """A binary string.
4290+
4291+ If the value is a Python3 str (unicode), it will be automatically
4292+ encoded.
4293+ """
4294 def coerce(self, value):
4295- if not isinstance(value, bytes):
4296- raise InvalidError("%r isn't a bytestring" % (value,))
4297- return value
4298+ if isinstance(value, bytes):
4299+ return value
4300+
4301+ if isinstance(value, str):
4302+ return value.encode()
4303+
4304+ raise InvalidError("%r isn't a bytestring" % value)
4305
4306
4307 class Unicode(object):
4308diff --git a/landscape/lib/testing.py b/landscape/lib/testing.py
4309index c024055..101ca97 100644
4310--- a/landscape/lib/testing.py
4311+++ b/landscape/lib/testing.py
4312@@ -15,7 +15,7 @@ import unittest
4313 from logging import Handler, ERROR, Formatter
4314 from twisted.trial.unittest import TestCase
4315 from twisted.python.compat import StringType as basestring
4316-from twisted.python.compat import _PY3
4317+from landscape.lib.compat import _PY3
4318 from twisted.python.failure import Failure
4319 from twisted.internet.defer import Deferred
4320 from twisted.internet.error import ConnectError
4321diff --git a/landscape/lib/tests/test_backoff.py b/landscape/lib/tests/test_backoff.py
4322new file mode 100644
4323index 0000000..18de645
4324--- /dev/null
4325+++ b/landscape/lib/tests/test_backoff.py
4326@@ -0,0 +1,37 @@
4327+from landscape.lib.backoff import ExponentialBackoff
4328+from landscape.client.tests.helpers import LandscapeTest
4329+
4330+
4331+class TestBackoff(LandscapeTest):
4332+
4333+ def test_increase(self):
4334+ '''Test the delay values start correctly and double'''
4335+ backoff_counter = ExponentialBackoff(5, 10)
4336+ backoff_counter.increase()
4337+ self.assertEqual(backoff_counter.get_delay(), 5)
4338+ backoff_counter.increase()
4339+ self.assertEqual(backoff_counter.get_delay(), 10)
4340+
4341+ def test_min(self):
4342+ '''Test the count and the delay never go below zero'''
4343+ backoff_counter = ExponentialBackoff(1, 5)
4344+ for _ in range(10):
4345+ backoff_counter.decrease()
4346+ self.assertEqual(backoff_counter.get_delay(), 0)
4347+ self.assertEqual(backoff_counter.get_random_delay(), 0)
4348+ self.assertEqual(backoff_counter._error_count, 0)
4349+
4350+ def test_max(self):
4351+ '''Test the delay never goes above max'''
4352+ backoff_counter = ExponentialBackoff(1, 5)
4353+ for _ in range(10):
4354+ backoff_counter.increase()
4355+ self.assertEqual(backoff_counter.get_delay(), 5)
4356+
4357+ def test_decreased_when_maxed(self):
4358+ '''Test the delay goes down one step when maxed'''
4359+ backoff_counter = ExponentialBackoff(1, 5)
4360+ for _ in range(10):
4361+ backoff_counter.increase()
4362+ backoff_counter.decrease()
4363+ self.assertTrue(backoff_counter.get_delay() < 5)
4364diff --git a/landscape/lib/tests/test_config.py b/landscape/lib/tests/test_config.py
4365index ee48727..3fbee3f 100644
4366--- a/landscape/lib/tests/test_config.py
4367+++ b/landscape/lib/tests/test_config.py
4368@@ -338,8 +338,10 @@ class BaseConfigurationTest(ConfigTestCase, HelperTestCase, unittest.TestCase):
4369 """))
4370 self.config.load_configuration_file(filename)
4371 self.assertEqual(self.config.whatever, "spam")
4372- self.assertIn("WARNING: Duplicate keyword name at line 4.",
4373- self.logfile.getvalue())
4374+ self.assertIn(
4375+ f"WARNING: ERROR at {filename}: Duplicate keyword name at line 4.",
4376+ self.logfile.getvalue(),
4377+ )
4378
4379 def test_duplicate_key(self):
4380 """
4381@@ -354,8 +356,10 @@ class BaseConfigurationTest(ConfigTestCase, HelperTestCase, unittest.TestCase):
4382 filename = self.makeFile(config)
4383 self.config.load_configuration_file(filename)
4384 self.assertEqual(self.config.computer_title, "frog")
4385- self.assertIn("WARNING: Duplicate keyword name at line 4.",
4386- self.logfile.getvalue())
4387+ self.assertIn(
4388+ f"WARNING: ERROR at {filename}: Duplicate keyword name at line 4.",
4389+ self.logfile.getvalue(),
4390+ )
4391
4392 def test_triplicate_key(self):
4393 """
4394@@ -372,8 +376,13 @@ class BaseConfigurationTest(ConfigTestCase, HelperTestCase, unittest.TestCase):
4395 self.config.load_configuration_file(filename)
4396 self.assertEqual(self.config.computer_title, "frog")
4397 logged = self.logfile.getvalue()
4398- self.assertIn("WARNING: Parsing failed with several errors.",
4399- logged)
4400+ self.assertIn(
4401+ (
4402+ f"WARNING: ERROR at {filename}: Parsing failed with several "
4403+ "errors."
4404+ ),
4405+ logged,
4406+ )
4407 self.assertIn("First error at line 4.", logged)
4408
4409 def test_load_not_found_default_accept_missing(self):
4410diff --git a/landscape/lib/tests/test_gpg.py b/landscape/lib/tests/test_gpg.py
4411index e6165a2..4c84e00 100644
4412--- a/landscape/lib/tests/test_gpg.py
4413+++ b/landscape/lib/tests/test_gpg.py
4414@@ -1,9 +1,10 @@
4415 import mock
4416 import os
4417+import textwrap
4418 import unittest
4419
4420 from twisted.internet import reactor
4421-from twisted.internet.defer import Deferred
4422+from twisted.internet.defer import Deferred, inlineCallbacks
4423
4424 from landscape.lib import testing
4425 from landscape.lib.gpg import gpg_verify
4426@@ -16,6 +17,8 @@ class GPGTest(testing.FSTestCase, testing.TwistedTestCase, unittest.TestCase):
4427 L{gpg_verify} runs the given gpg binary and returns C{True} if the
4428 provided signature is valid.
4429 """
4430+ aptdir = self.makeDir()
4431+ os.mknod("{}/trusted.gpg".format(aptdir))
4432 gpg_options = self.makeFile()
4433 gpg = self.makeFile("#!/bin/sh\n"
4434 "touch $3/trustdb.gpg\n"
4435@@ -27,14 +30,15 @@ class GPGTest(testing.FSTestCase, testing.TwistedTestCase, unittest.TestCase):
4436 @mock.patch("tempfile.mkdtemp")
4437 def do_test(mkdtemp_mock):
4438 mkdtemp_mock.return_value = gpg_home
4439- result = gpg_verify("/some/file", "/some/signature", gpg=gpg)
4440+ result = gpg_verify(
4441+ "/some/file", "/some/signature", gpg=gpg, apt_dir=aptdir)
4442
4443 def check_result(ignored):
4444 self.assertEqual(
4445 open(gpg_options).read(),
4446 "--no-options --homedir %s --no-default-keyring "
4447- "--ignore-time-conflict --keyring /etc/apt/trusted.gpg "
4448- "--verify /some/signature /some/file" % gpg_home)
4449+ "--ignore-time-conflict --keyring %s/trusted.gpg "
4450+ "--verify /some/signature /some/file" % (gpg_home, aptdir))
4451 self.assertFalse(os.path.exists(gpg_home))
4452
4453 result.addCallback(check_result)
4454@@ -70,3 +74,38 @@ class GPGTest(testing.FSTestCase, testing.TwistedTestCase, unittest.TestCase):
4455
4456 reactor.callWhenRunning(do_test)
4457 return deferred
4458+
4459+ @inlineCallbacks
4460+ def test_gpg_verify_trusted_dir(self):
4461+ """
4462+ gpg_verify uses keys from the trusted.gpg.d if such a folder exists.
4463+ """
4464+ apt_dir = self.makeDir()
4465+ os.mkdir("{}/trusted.gpg.d".format(apt_dir))
4466+ os.mknod("{}/trusted.gpg.d/foo.gpg".format(apt_dir))
4467+ os.mknod("{}/trusted.gpg.d/baz.gpg".format(apt_dir))
4468+ os.mknod("{}/trusted.gpg.d/bad.gpg~".format(apt_dir))
4469+
4470+ gpg_call = self.makeFile()
4471+ fake_gpg = self.makeFile(textwrap.dedent("""\
4472+ #!/bin/sh
4473+ touch $3/trustdb.gpg
4474+ echo -n $@ > {}
4475+ """).format(gpg_call))
4476+ os.chmod(fake_gpg, 0o755)
4477+ gpg_home = self.makeDir()
4478+
4479+ with mock.patch("tempfile.mkdtemp", return_value=gpg_home):
4480+ yield gpg_verify(
4481+ "/some/file", "/some/signature", gpg=fake_gpg, apt_dir=apt_dir)
4482+
4483+ expected = (
4484+ "--no-options --homedir {gpg_home} --no-default-keyring "
4485+ "--ignore-time-conflict "
4486+ "--keyring {apt_dir}/trusted.gpg.d/baz.gpg "
4487+ "--keyring {apt_dir}/trusted.gpg.d/foo.gpg "
4488+ "--verify /some/signature /some/file"
4489+ ).format(gpg_home=gpg_home, apt_dir=apt_dir)
4490+ with open(gpg_call) as call:
4491+ self.assertEqual(expected, call.read())
4492+ self.assertFalse(os.path.exists(gpg_home))
4493diff --git a/landscape/lib/tests/test_lsb_release.py b/landscape/lib/tests/test_lsb_release.py
4494index c1a23a6..da1932d 100644
4495--- a/landscape/lib/tests/test_lsb_release.py
4496+++ b/landscape/lib/tests/test_lsb_release.py
4497@@ -1,4 +1,6 @@
4498 import unittest
4499+from subprocess import CalledProcessError
4500+from unittest import mock
4501
4502 from landscape.lib import testing
4503 from landscape.lib.lsb_release import parse_lsb_release
4504@@ -7,6 +9,30 @@ from landscape.lib.lsb_release import parse_lsb_release
4505 class LsbReleaseTest(testing.FSTestCase, unittest.TestCase):
4506
4507 def test_parse_lsb_release(self):
4508+ with mock.patch("landscape.lib.lsb_release.check_output") as co_mock:
4509+ co_mock.return_value = (b"Ubuntu\nUbuntu 22.04.1 LTS\n22.04\njammy"
4510+ b"\n")
4511+ lsb_release = parse_lsb_release()
4512+
4513+ self.assertEqual(lsb_release,
4514+ {"distributor-id": "Ubuntu",
4515+ "description": "Ubuntu 22.04.1 LTS",
4516+ "release": "22.04",
4517+ "code-name": "jammy"})
4518+
4519+ def test_parse_lsb_release_debian(self):
4520+ with mock.patch("landscape.lib.lsb_release.check_output") as co_mock:
4521+ co_mock.return_value = (b"Debian\nDebian GNU/Linux 11 (bullseye)\n"
4522+ b"11\nbullseye\n")
4523+ lsb_release = parse_lsb_release()
4524+
4525+ self.assertEqual(lsb_release,
4526+ {"distributor-id": "Debian",
4527+ "description": "Debian GNU/Linux 11 (bullseye)",
4528+ "release": "11",
4529+ "code-name": "bullseye"})
4530+
4531+ def test_parse_lsb_release_file(self):
4532 """
4533 L{parse_lsb_release} returns a C{dict} holding information from
4534 the given LSB release file.
4535@@ -17,18 +43,33 @@ class LsbReleaseTest(testing.FSTestCase, unittest.TestCase):
4536 "DISTRIB_DESCRIPTION="
4537 "\"Ubuntu 6.06.1 LTS\"\n")
4538
4539- self.assertEqual(parse_lsb_release(lsb_release_filename),
4540+ with mock.patch("landscape.lib.lsb_release.check_output") as co_mock:
4541+ co_mock.side_effect = CalledProcessError(127, "")
4542+ lsb_release = parse_lsb_release(lsb_release_filename)
4543+
4544+ self.assertEqual(lsb_release,
4545 {"distributor-id": "Ubuntu",
4546 "description": "Ubuntu 6.06.1 LTS",
4547 "release": "6.06",
4548 "code-name": "dapper"})
4549
4550- def test_parse_lsb_release_with_missing_or_extra_fields(self):
4551+ def test_parse_lsb_release_file_with_missing_or_extra_fields(self):
4552 """
4553 L{parse_lsb_release} ignores lines not matching the map of
4554 known keys, and returns only keys with an actual value.
4555 """
4556 lsb_release_filename = self.makeFile("DISTRIB_ID=Ubuntu\n"
4557 "FOO=Bar\n")
4558- self.assertEqual(parse_lsb_release(lsb_release_filename),
4559- {"distributor-id": "Ubuntu"})
4560+
4561+ with mock.patch("landscape.lib.lsb_release.check_output") as co_mock:
4562+ co_mock.side_effect = CalledProcessError(127, "")
4563+ lsb_release = parse_lsb_release(lsb_release_filename)
4564+
4565+ self.assertEqual(lsb_release, {"distributor-id": "Ubuntu"})
4566+
4567+ def test_parse_lsb_release_file_not_found(self):
4568+ with mock.patch("landscape.lib.lsb_release.check_output") as co_mock:
4569+ co_mock.side_effect = CalledProcessError(127, "")
4570+
4571+ self.assertRaises(FileNotFoundError, parse_lsb_release,
4572+ "TheresNoWayThisFileExists")
4573diff --git a/landscape/lib/tests/test_network.py b/landscape/lib/tests/test_network.py
4574index 0b56971..27d2e20 100644
4575--- a/landscape/lib/tests/test_network.py
4576+++ b/landscape/lib/tests/test_network.py
4577@@ -1,7 +1,7 @@
4578 import socket
4579 import unittest
4580
4581-from mock import patch, ANY, mock_open
4582+from unittest.mock import ANY, DEFAULT, patch, mock_open
4583 from netifaces import (
4584 AF_INET,
4585 AF_INET6,
4586@@ -14,8 +14,8 @@ from subprocess import Popen, PIPE
4587
4588 from landscape.lib import testing
4589 from landscape.lib.network import (
4590- get_network_traffic, get_active_device_info, get_active_interfaces,
4591- get_fqdn, get_network_interface_speed, is_up)
4592+ get_active_device_info, get_filtered_if_info, get_fqdn,
4593+ get_network_interface_speed, get_network_traffic, is_up)
4594
4595
4596 class BaseTestCase(testing.HelperTestCase, unittest.TestCase):
4597@@ -148,33 +148,62 @@ class NetworkInfoTest(BaseTestCase):
4598 'duplex': True,
4599 'ip_addresses': {AF_INET6: [{'addr': '2001::1'}]}}])
4600
4601+ @patch("landscape.lib.network.get_default_interfaces")
4602+ @patch("landscape.lib.network.get_network_interface_speed")
4603+ @patch("landscape.lib.network.get_flags")
4604+ @patch("landscape.lib.network.netifaces.ifaddresses")
4605+ @patch("landscape.lib.network.netifaces.interfaces")
4606+ def test_default_only_interface(
4607+ self, mock_interfaces, mock_ifaddresses, mock_get_flags,
4608+ mock_get_network_interface_speed, mock_get_default_interfaces):
4609+ default_iface = "test_iface"
4610+ mock_get_default_interfaces.return_value = [default_iface]
4611+ mock_get_network_interface_speed.return_value = (100, True)
4612+ mock_get_flags.return_value = 4163
4613+ mock_interfaces.return_value = [default_iface]
4614+ mock_ifaddresses.return_value = {AF_INET: [{"addr": "192.168.0.50"}]}
4615+ device_info = get_active_device_info(extended=False, default_only=True)
4616+ interfaces = [i["interface"] for i in device_info]
4617+ self.assertIn(default_iface, interfaces)
4618+ self.assertEqual(len(interfaces), 1)
4619+
4620 def test_skip_loopback(self):
4621 """The C{lo} interface is not reported by L{get_active_device_info}."""
4622 device_info = get_active_device_info()
4623 interfaces = [i["interface"] for i in device_info]
4624 self.assertNotIn("lo", interfaces)
4625
4626- @patch("landscape.lib.network.get_active_interfaces")
4627- def test_skip_vlan(self, mock_get_active_interfaces):
4628+ @patch("landscape.lib.network.netifaces.interfaces")
4629+ def test_skip_vlan(self, mock_interfaces):
4630 """VLAN interfaces are not reported by L{get_active_device_info}."""
4631- mock_get_active_interfaces.side_effect = lambda: (
4632- list(get_active_interfaces()) + [("eth0.1", {})])
4633+ mock_interfaces.side_effect = lambda: (
4634+ _interfaces() + ["eth0.1"])
4635 device_info = get_active_device_info()
4636- self.assertTrue(mock_get_active_interfaces.called)
4637 interfaces = [i["interface"] for i in device_info]
4638 self.assertNotIn("eth0.1", interfaces)
4639
4640- @patch("landscape.lib.network.get_active_interfaces")
4641- def test_skip_alias(self, mock_get_active_interfaces):
4642+ @patch("landscape.lib.network.netifaces.interfaces")
4643+ def test_skip_alias(self, mock_interfaces):
4644 """Interface aliases are not reported by L{get_active_device_info}."""
4645- mock_get_active_interfaces.side_effect = lambda: (
4646- list(get_active_interfaces()) + [("eth0:foo", {})])
4647+ mock_interfaces.side_effect = lambda: (
4648+ _interfaces() + ["eth0:foo"])
4649 device_info = get_active_device_info()
4650 interfaces = [i["interface"] for i in device_info]
4651 self.assertNotIn("eth0:foo", interfaces)
4652
4653 @patch("landscape.lib.network.netifaces.ifaddresses")
4654 @patch("landscape.lib.network.netifaces.interfaces")
4655+ def test_no_extra_netifaces_calls(self, mock_interfaces, mock_ifaddresses):
4656+ """
4657+ Make sure filtered out interfaces aren't used in netiface calls due to
4658+ their impact on sysinfo/login time with a large amount of interfaces.
4659+ """
4660+ mock_interfaces.return_value = ["eth0:foo"]
4661+ get_active_device_info()
4662+ assert not mock_ifaddresses.called
4663+
4664+ @patch("landscape.lib.network.netifaces.ifaddresses")
4665+ @patch("landscape.lib.network.netifaces.interfaces")
4666 def test_skip_iface_with_no_addr(self, mock_interfaces, mock_ifaddresses):
4667 mock_interfaces.return_value = _interfaces() + ["test_iface"]
4668 mock_ifaddresses.side_effect = lambda iface: (
4669@@ -283,6 +312,53 @@ class NetworkInfoTest(BaseTestCase):
4670 self.assertFalse(is_up(2 + 64 + 4096))
4671 self.assertFalse(is_up(0b11111111111110))
4672
4673+ def test_get_filtered_if_info(self):
4674+ def filter_tap(interface):
4675+ return interface.startswith("tap")
4676+
4677+ with patch.multiple(
4678+ "landscape.lib.network",
4679+ get_flags=DEFAULT,
4680+ get_network_interface_speed=DEFAULT,
4681+ netifaces=DEFAULT,
4682+ ) as mocks:
4683+ mocks["netifaces"].interfaces.return_value = [
4684+ "tap0123",
4685+ "test_iface",
4686+ "tap4567",
4687+ "test_iface2",
4688+ ]
4689+ mocks["get_flags"].return_value = 4163
4690+ mocks["get_network_interface_speed"].return_value = (100, True)
4691+ device_info = get_filtered_if_info(filters=(filter_tap,),
4692+ extended=True)
4693+
4694+ self.assertEqual(len(device_info), 2)
4695+ self.assertTrue(all("tap" not in i["interface"] for i in device_info))
4696+
4697+ def test_get_active_device_info_filtered_taps(self):
4698+ """
4699+ Tests that tap network interfaces are filtered out.
4700+ """
4701+ with patch.multiple(
4702+ "landscape.lib.network",
4703+ get_flags=DEFAULT,
4704+ get_network_interface_speed=DEFAULT,
4705+ netifaces=DEFAULT,
4706+ ) as mocks:
4707+ mocks["netifaces"].interfaces.return_value = [
4708+ "tap0123",
4709+ "test_iface",
4710+ "tap4567",
4711+ "test_iface2",
4712+ ]
4713+ mocks["get_flags"].return_value = 4163
4714+ mocks["get_network_interface_speed"].return_value = (100, True)
4715+ device_info = get_active_device_info(extended=True)
4716+
4717+ self.assertEqual(len(device_info), 2)
4718+ self.assertTrue(all("tap" not in i["interface"] for i in device_info))
4719+
4720
4721 # exact output of cat /proc/net/dev snapshot with line continuations for pep8
4722 test_proc_net_dev_output = """\
4723@@ -370,6 +446,7 @@ class NetworkInterfaceSpeedTest(BaseTestCase):
4724 mock_unpack.assert_called_with("12xHB28x", ANY)
4725
4726 self.assertEqual((100, False), result)
4727+ sock.close()
4728
4729 @patch("struct.unpack")
4730 @patch("fcntl.ioctl")
4731@@ -389,6 +466,7 @@ class NetworkInterfaceSpeedTest(BaseTestCase):
4732 mock_unpack.assert_called_with("12xHB28x", ANY)
4733
4734 self.assertEqual((0, False), result)
4735+ sock.close()
4736
4737 @patch("fcntl.ioctl")
4738 def test_get_network_interface_speed_not_supported(self, mock_ioctl):
4739@@ -411,6 +489,7 @@ class NetworkInterfaceSpeedTest(BaseTestCase):
4740 mock_ioctl.assert_called_with(ANY, ANY, ANY)
4741
4742 self.assertEqual((-1, False), result)
4743+ sock.close()
4744
4745 @patch("fcntl.ioctl")
4746 def test_get_network_interface_speed_not_permitted(self, mock_ioctl):
4747@@ -433,6 +512,7 @@ class NetworkInterfaceSpeedTest(BaseTestCase):
4748 mock_ioctl.assert_called_with(ANY, ANY, ANY)
4749
4750 self.assertEqual((-1, False), result)
4751+ sock.close()
4752
4753 @patch("fcntl.ioctl")
4754 def test_get_network_interface_speed_other_io_error(self, mock_ioctl):
4755@@ -450,3 +530,4 @@ class NetworkInterfaceSpeedTest(BaseTestCase):
4756 mock_ioctl.side_effect = theerror
4757
4758 self.assertRaises(IOError, get_network_interface_speed, sock, b"eth0")
4759+ sock.close()
4760diff --git a/landscape/lib/tests/test_schema.py b/landscape/lib/tests/test_schema.py
4761index cd6d71f..fce6979 100644
4762--- a/landscape/lib/tests/test_schema.py
4763+++ b/landscape/lib/tests/test_schema.py
4764@@ -69,8 +69,8 @@ class BasicTypesTest(unittest.TestCase):
4765 def test_string(self):
4766 self.assertEqual(Bytes().coerce(b"foo"), b"foo")
4767
4768- def test_string_bad_unicode(self):
4769- self.assertRaises(InvalidError, Bytes().coerce, u"foo")
4770+ def test_string_unicode(self):
4771+ self.assertEqual(Bytes().coerce(u"foo"), b"foo")
4772
4773 def test_string_bad_anything(self):
4774 self.assertRaises(InvalidError, Bytes().coerce, object())
4775diff --git a/landscape/lib/tests/test_vm_info.py b/landscape/lib/tests/test_vm_info.py
4776index 68bf6a3..3e46912 100644
4777--- a/landscape/lib/tests/test_vm_info.py
4778+++ b/landscape/lib/tests/test_vm_info.py
4779@@ -165,9 +165,9 @@ class GetVMInfoTest(BaseTestCase):
4780 """
4781 cpuinfo_path = os.path.join(self.proc_path, "cpuinfo")
4782 cpuinfo = (
4783- "platform : Some Machine\n"
4784- "model : Some CPU (emulated by qemu)\n"
4785- "machine : Some Machine (emulated by qemu)\n")
4786+ "platform : Some Machine\n"
4787+ "model : Some CPU (emulated by qemu)\n"
4788+ "machine : Some Machine (emulated by qemu)\n")
4789 self.makeFile(path=cpuinfo_path, content=cpuinfo)
4790 self.assertEqual(b"kvm", get_vm_info(root_path=self.root_path))
4791
4792@@ -184,6 +184,18 @@ class GetVMInfoTest(BaseTestCase):
4793 self.make_dmi_info("product_name", "KVM")
4794 self.assertEqual(b"kvm", get_vm_info(root_path=self.root_path))
4795
4796+ def test_get_vm_info_with_rhev(self):
4797+ """get_vm_info returns 'kvm' if running under RHEV Hypervisor."""
4798+ self.make_dmi_info("product_name", "RHEV Hypervisor")
4799+ self.make_dmi_info("sys_vendor", "Red Hat")
4800+ self.assertEqual(b"kvm", get_vm_info(root_path=self.root_path))
4801+
4802+ def test_get_vm_info_with_parallels(self):
4803+ """get_vm_info returns 'kvm' if running under Parallels"""
4804+ self.make_dmi_info("product_name", "Parallels Virtual Platform")
4805+ self.make_dmi_info("sys_vendor", "Parallels Software International")
4806+ self.assertEqual(b"kvm", get_vm_info(root_path=self.root_path))
4807+
4808
4809 class GetContainerInfoTest(BaseTestCase):
4810
4811diff --git a/landscape/lib/user.py b/landscape/lib/user.py
4812index db8a24f..dfac4f9 100644
4813--- a/landscape/lib/user.py
4814+++ b/landscape/lib/user.py
4815@@ -1,7 +1,7 @@
4816 import os.path
4817 import pwd
4818
4819-from twisted.python.compat import _PY3
4820+from landscape.lib.compat import _PY3
4821
4822 from landscape.lib.encoding import encode_if_needed
4823
4824diff --git a/landscape/lib/vm_info.py b/landscape/lib/vm_info.py
4825index 2ede13b..a5d7ca7 100644
4826--- a/landscape/lib/vm_info.py
4827+++ b/landscape/lib/vm_info.py
4828@@ -66,9 +66,6 @@ def _get_vm_by_vendor(sys_vendor_path):
4829 # We need bytes here as required by the message schema.
4830 vendor = read_binary_file(sys_vendor_path, limit=1024).lower()
4831
4832- # 2018-01: AWS and DO are now returning custom sys_vendor names
4833- # instead of qemu. If this becomes a trend, it may be worth also checking
4834- # dmi/id/chassis_vendor which seems to unchanged (bochs).
4835 content_vendors_map = (
4836 (b"amazon ec2", b"kvm"),
4837 (b"bochs", b"kvm"),
4838@@ -81,6 +78,8 @@ def _get_vm_by_vendor(sys_vendor_path):
4839 (b"qemu", b"kvm"),
4840 (b"kvm", b"kvm"),
4841 (b"vmware", b"vmware"),
4842+ (b"rhev", b"kvm"),
4843+ (b"parallels", b"kvm")
4844 )
4845 for name, vm_type in content_vendors_map:
4846 if name in vendor:
4847diff --git a/landscape/message_schemas/server_bound.py b/landscape/message_schemas/server_bound.py
4848index a03d76c..1e8f698 100644
4849--- a/landscape/message_schemas/server_bound.py
4850+++ b/landscape/message_schemas/server_bound.py
4851@@ -19,7 +19,7 @@ __all__ = [
4852 "NETWORK_DEVICE", "NETWORK_ACTIVITY",
4853 "REBOOT_REQUIRED_INFO", "UPDATE_MANAGER_INFO", "CPU_USAGE",
4854 "CEPH_USAGE", "SWIFT_USAGE", "SWIFT_DEVICE_INFO", "KEYSTONE_TOKEN",
4855- "JUJU_UNITS_INFO", "CLOUD_METADATA",
4856+ "JUJU_UNITS_INFO", "CLOUD_METADATA", "COMPUTER_TAGS", "UBUNTU_PRO_INFO",
4857 ]
4858
4859
4860@@ -219,10 +219,13 @@ REGISTER_3_3 = Message(
4861 "juju-info": KeyDict({"environment-uuid": Unicode(),
4862 "api-addresses": List(Unicode()),
4863 "machine-id": Unicode()}),
4864- "access_group": Unicode()},
4865+ "access_group": Unicode(),
4866+ "clone_secure_id": Any(Unicode(), Constant(None)),
4867+ "ubuntu_pro_info": Unicode()},
4868 api=b"3.3",
4869 optional=["registration_password", "hostname", "tags", "vm-info",
4870- "container-info", "access_group", "juju-info"])
4871+ "container-info", "access_group", "juju-info",
4872+ "clone_secure_id", "ubuntu_pro_info"])
4873
4874
4875 # XXX The register-provisioned-machine message is obsolete, it's kept around
4876@@ -504,6 +507,13 @@ NETWORK_ACTIVITY = Message(
4877
4878 UPDATE_MANAGER_INFO = Message("update-manager-info", {"prompt": Unicode()})
4879
4880+COMPUTER_TAGS = Message(
4881+ "computer-tags",
4882+ {"tags": Any(Unicode(), Constant(None))})
4883+
4884+UBUNTU_PRO_INFO = Message(
4885+ "ubuntu-pro-info",
4886+ {"ubuntu-pro-info": Unicode()})
4887
4888 message_schemas = (
4889 ACTIVE_PROCESS_INFO, COMPUTER_UPTIME, CLIENT_UPTIME,
4890@@ -518,4 +528,4 @@ message_schemas = (
4891 NETWORK_DEVICE, NETWORK_ACTIVITY,
4892 REBOOT_REQUIRED_INFO, UPDATE_MANAGER_INFO, CPU_USAGE,
4893 CEPH_USAGE, SWIFT_USAGE, SWIFT_DEVICE_INFO, KEYSTONE_TOKEN,
4894- JUJU_UNITS_INFO, CLOUD_METADATA)
4895+ JUJU_UNITS_INFO, CLOUD_METADATA, COMPUTER_TAGS, UBUNTU_PRO_INFO)
4896diff --git a/landscape/message_schemas/test_message.py b/landscape/message_schemas/test_message.py
4897index 0d87eac..7b8c66b 100644
4898--- a/landscape/message_schemas/test_message.py
4899+++ b/landscape/message_schemas/test_message.py
4900@@ -1,6 +1,6 @@
4901 import unittest
4902
4903-from landscape.lib.schema import Int
4904+from landscape.lib.schema import Constant, Int
4905 from landscape.message_schemas.message import Message
4906
4907
4908@@ -13,6 +13,14 @@ class MessageTest(unittest.TestCase):
4909 schema.coerce({"type": "foo", "data": 3}),
4910 {"type": "foo", "data": 3})
4911
4912+ def test_coerce_bytes_to_str(self):
4913+ """
4914+ The L{Constant} schema type recognizes bytestrings that decode to
4915+ matching strings.
4916+ """
4917+ constant = Constant("register")
4918+ self.assertEqual(constant.coerce(b"register"), "register")
4919+
4920 def test_timestamp(self):
4921 """L{Message} schemas should accept C{timestamp} keys."""
4922 schema = Message("bar", {})
4923diff --git a/landscape/sysinfo/load.py b/landscape/sysinfo/load.py
4924index 718a929..4843038 100644
4925--- a/landscape/sysinfo/load.py
4926+++ b/landscape/sysinfo/load.py
4927@@ -9,5 +9,6 @@ class Load(object):
4928 self._sysinfo = sysinfo
4929
4930 def run(self):
4931- self._sysinfo.add_header("System load", str(os.getloadavg()[0]))
4932+ self._sysinfo.add_header(
4933+ "System load", str(round(os.getloadavg()[0], 2)))
4934 return succeed(None)
4935diff --git a/landscape/sysinfo/network.py b/landscape/sysinfo/network.py
4936index 6df4b7c..420a603 100644
4937--- a/landscape/sysinfo/network.py
4938+++ b/landscape/sysinfo/network.py
4939@@ -16,7 +16,8 @@ class Network(object):
4940
4941 def __init__(self, get_device_info=None):
4942 if get_device_info is None:
4943- get_device_info = partial(get_active_device_info, extended=True)
4944+ get_device_info = partial(get_active_device_info,
4945+ extended=True, default_only=True)
4946 self._get_device_info = get_device_info
4947
4948 def register(self, sysinfo):
4949diff --git a/man/landscape-config.1 b/man/landscape-config.1
4950index 7d398ff..eeafefd 100644
4951--- a/man/landscape-config.1
4952+++ b/man/landscape-config.1
4953@@ -1,5 +1,5 @@
4954 .\" Text automatically generated by txt2man
4955-.TH landscape-config 1 "14 February 2019" "" ""
4956+.TH landscape-config 1 "07 February 2022" "" ""
4957 .SH NAME
4958 \fBlandscape-config \fP- configure the Landscape management client
4959 \fB
4960@@ -181,6 +181,12 @@ Stop running clients and disable start at boot.
4961 .B
4962 \fB--otp\fP=OTP
4963 The one-time password (OTP) to use in cloud configuration.
4964+.TP
4965+.B
4966+\fB--is-registered\fP
4967+Exit with code 0 (success) if client
4968+is registered else returns 5. Display
4969+registration info.
4970 .SH CLOUD
4971
4972 Landscape has some cloud features that become available when the EC2 or
4973diff --git a/man/landscape-config.txt b/man/landscape-config.txt
4974index 5a100bf..2bce364 100644
4975--- a/man/landscape-config.txt
4976+++ b/man/landscape-config.txt
4977@@ -75,6 +75,9 @@ OPTIONS
4978 --silent Run without manual interaction.
4979 --disable Stop running clients and disable start at boot.
4980 --otp=OTP The one-time password (OTP) to use in cloud configuration.
4981+ --is-registered Exit with code 0 (success) if client
4982+ is registered else returns 5. Display
4983+ registration info.
4984
4985 CLOUD
4986

Subscribers

People subscribed via source and target branches