Merge ~sylvain-pineau/plainbox:xmas_diet into plainbox:master

Proposed by Sylvain Pineau
Status: Merged
Approved by: Sylvain Pineau
Approved revision: b9e6cf83e09bc42c03ed9749c9dd149e7c86620c
Merged at revision: 638986c4a11114b188cbc61f754ebafebf78b36a
Proposed branch: ~sylvain-pineau/plainbox:xmas_diet
Merge into: plainbox:master
Diff against target: 19011 lines (+217/-3860)
78 files modified
MANIFEST.in (+0/-1)
dev/null (+0/-459)
docs/author/index.rst (+0/-2)
docs/author/intro.rst (+4/-169)
docs/author/provider-files.rst (+0/-6)
docs/author/provider-namespaces.rst (+4/-4)
docs/author/provider-template.rst (+7/-29)
docs/author/providers.rst (+1/-1)
docs/dev/old.rst (+0/-54)
docs/dev/trusted-launcher.rst (+5/-9)
docs/glossary.rst (+5/-6)
docs/manpages/plainbox-dev-analyze.rst (+5/-12)
docs/manpages/plainbox-exporter-units.rst (+1/-33)
docs/manpages/plainbox-file-units.rst (+1/-4)
docs/manpages/plainbox-job-units.rst (+0/-3)
docs/manpages/plainbox-run.rst (+5/-75)
docs/manpages/plainbox-test-plan-units.rst (+5/-6)
docs/manpages/plainbox-trusted-launcher-1.rst (+1/-6)
docs/usage.rst (+2/-13)
plainbox/__init__.py (+2/-8)
plainbox/abc.py (+4/-22)
plainbox/impl/applogic.py (+0/-20)
plainbox/impl/buildsystems.py (+0/-7)
plainbox/impl/commands/__init__.py (+1/-1)
plainbox/impl/commands/cmd_analyze.py (+0/-11)
plainbox/impl/commands/cmd_checkbox.py (+0/-9)
plainbox/impl/commands/inv_analyze.py (+0/-46)
plainbox/impl/commands/inv_checkbox.py (+0/-52)
plainbox/impl/commands/inv_run.py (+4/-9)
plainbox/impl/commands/inv_special.py (+0/-3)
plainbox/impl/commands/test_run.py (+3/-8)
plainbox/impl/ctrl.py (+3/-115)
plainbox/impl/depmgr.py (+1/-1)
plainbox/impl/exporter/__init__.py (+0/-9)
plainbox/impl/exporter/tar.py (+1/-2)
plainbox/impl/exporter/test_html.py (+1/-1)
plainbox/impl/exporter/test_init.py (+0/-2)
plainbox/impl/exporter/xlsx.py (+2/-48)
plainbox/impl/highlevel.py (+1/-6)
plainbox/impl/launcher.py (+1/-87)
plainbox/impl/providers/__init__.py (+1/-12)
plainbox/impl/providers/exporters/data/checkbox.html (+1/-1)
plainbox/impl/providers/exporters/data/checkbox.json (+1/-1)
plainbox/impl/providers/exporters/units/exporter.pxu (+1/-7)
plainbox/impl/providers/stubbox/units/jobs/representative.pxu (+0/-9)
plainbox/impl/providers/stubbox/units/jobs/stub.pxu (+0/-24)
plainbox/impl/result.py (+1/-25)
plainbox/impl/runner.py (+3/-40)
plainbox/impl/secure/launcher1.py (+5/-9)
plainbox/impl/secure/providers/__init__.py (+1/-12)
plainbox/impl/secure/providers/test_v1.py (+18/-94)
plainbox/impl/secure/providers/v1.py (+16/-203)
plainbox/impl/secure/qualifiers.py (+1/-223)
plainbox/impl/secure/test_launcher1.py (+10/-37)
plainbox/impl/secure/test_qualifiers.py (+0/-231)
plainbox/impl/session/assistant.py (+5/-6)
plainbox/impl/session/jobs.py (+1/-1)
plainbox/impl/session/manager.py (+8/-29)
plainbox/impl/session/state.py (+5/-16)
plainbox/impl/session/storage.py (+38/-471)
plainbox/impl/session/test_manager.py (+3/-3)
plainbox/impl/session/test_resume.py (+1/-302)
plainbox/impl/session/test_state.py (+3/-64)
plainbox/impl/session/test_storage.py (+4/-71)
plainbox/impl/session/test_suspend.py (+0/-217)
plainbox/impl/test_box.py (+0/-67)
plainbox/impl/test_ctrl.py (+4/-134)
plainbox/impl/test_launcher.py (+2/-24)
plainbox/impl/unit/concrete_validators.py (+0/-5)
plainbox/impl/unit/file.py (+0/-1)
plainbox/impl/unit/job.py (+2/-21)
plainbox/impl/unit/test_job.py (+1/-12)
plainbox/impl/unit/testplan.py (+3/-3)
plainbox/impl/xparsers.py (+0/-98)
plainbox/impl/xscanners.py (+1/-1)
plainbox/provider_manager.py (+5/-19)
plainbox/test_provider_manager.py (+1/-3)
setup.py (+1/-5)
Reviewer Review Type Date Requested Status
Devices Certification Bot Needs Fixing
Maciej Kisielewski Approve
Review via email: mp+335580@code.launchpad.net

Description of the change

Cleanup of obsolete code (whitelists, local jobs, py3.2)

To post a comment you must log in.
Revision history for this message
Maciej Kisielewski (kissiel) wrote :

Great branch. Also literally :)

Few observations:

Saving storage now uses the py33 method that uses python API for 'saving-at-location'. This is necessary for corner cases when saving. This is the only thing that makes checkbox not run on OSX at the moment.

Lazy modules were not actually used properly, as many of the modules are imported at checkbox start-up, so their laziness was not actually employed.

Overall, awesome stuff covered in awesomesauce. Great christmas present!
Huge +1!

review: Approve
Revision history for this message
Devices Certification Bot (ce-certification-qa) wrote :
Download full text (4.8 KiB)

The merge was fine but running tests failed.

[trusty] starting container
[trusty] (timing) 0.00user 0.00system 0:00.14elapsed 5%CPU (0avgtext+0avgdata 4960maxresident)k
[trusty] (timing) 432inputs+72outputs (1major+1436minor)pagefaults 0swaps
[trusty] provisioning container
[trusty] (timing) 38.72user 12.08system 2:06.54elapsed 40%CPU (0avgtext+0avgdata 68076maxresident)k
[trusty] (timing) 591960inputs+1882064outputs (581major+1356518minor)pagefaults 0swaps
[trusty-testing] Starting tests...
Found a test script: ./requirements/001-container-tests-plainbox-egg-info
[trusty-testing] 001-container-tests-plainbox-egg-info: PASS
[trusty-testing] (timing) 0.29user 0.03system 0:00.34elapsed 94%CPU (0avgtext+0avgdata 22680maxresident)k
[trusty-testing] (timing) 760inputs+96outputs (17major+9248minor)pagefaults 0swaps
Found a test script: ./requirements/container-tests-plainbox
[trusty-testing] container-tests-plainbox: PASS
[trusty-testing] (timing) 37.84user 2.28system 0:45.87elapsed 87%CPU (0avgtext+0avgdata 68532maxresident)k
[trusty-testing] (timing) 18768inputs+82760outputs (71major+931471minor)pagefaults 0swaps
Found a test script: ./requirements/container-tests-plainbox-documentation
[trusty-testing] container-tests-plainbox-documentation: FAIL
[trusty-testing] stdout: https://paste.ubuntu.com/26345630/
[trusty-testing] stderr: https://paste.ubuntu.com/26345631/
[trusty-testing] (timing) Command exited with non-zero status 1
[trusty-testing] (timing) 1.40user 0.07system 0:01.55elapsed 94%CPU (0avgtext+0avgdata 52832maxresident)k
[trusty-testing] (timing) 11544inputs+1720outputs (17major+20042minor)pagefaults 0swaps
Found a test script: ./requirements/container-tests-plainbox-integration
[trusty-testing] container-tests-plainbox-integration: PASS
[trusty-testing] (timing) 0.91user 0.06system 0:01.02elapsed 95%CPU (0avgtext+0avgdata 39536maxresident)k
[trusty-testing] (timing) 56inputs+112outputs (0major+30120minor)pagefaults 0swaps
Found a test script: ./requirements/container-tests-providers-internal
[trusty-testing] container-tests-providers-internal: PASS
[trusty-testing] (timing) 1.12user 0.05system 0:01.20elapsed 98%CPU (0avgtext+0avgdata 30060maxresident)k
[trusty-testing] (timing) 264inputs+8outputs (1major+19705minor)pagefaults 0swaps
[trusty-testing] Fixing file permissions in source directory
[trusty-testing] Destroying container
Name: trusty-testing
State: STOPPED
[xenial] starting container
[xenial] (timing) 0.00user 0.00system 0:00.13elapsed 5%CPU (0avgtext+0avgdata 4984maxresident)k
[xenial] (timing) 496inputs+72outputs (1major+1427minor)pagefaults 0swaps
[xenial] provisioning container
[xenial] (timing) 42.40user 7.98system 1:44.10elapsed 48%CPU (0avgtext+0avgdata 81308maxresident)k
[xenial] (timing) 725800inputs+2195984outputs (594major+1640039minor)pagefaults 0swaps
[xenial-testing] Starting tests...
Found a test script: ./requirements/001-container-tests-plainbox-egg-info
[xenial-testing] 001-container-tests-plainbox-egg-info: PASS
[xenial-testing] (timing) 0.34user 0.04system 0:00.40elapsed 94%CPU (0avgtext+0avgdata 23860ma...

Read more...

review: Needs Fixing

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/MANIFEST.in b/MANIFEST.in
2index e51f963..1eb7c3b 100644
3--- a/MANIFEST.in
4+++ b/MANIFEST.in
5@@ -35,7 +35,6 @@ include plainbox/impl/providers/stubbox/po/*.pot
6 include plainbox/impl/providers/stubbox/po/POTFILES.in
7 include plainbox/impl/providers/stubbox/units/jobs/*.pxu
8 include plainbox/impl/providers/stubbox/units/testplans/*.pxu
9-include plainbox/impl/providers/stubbox/whitelists/*.whitelist
10
11 include plainbox/vendor/argparse/py*-argparse.py
12
13diff --git a/daily-package-testing/README b/daily-package-testing/README
14deleted file mode 100644
15index 27d3e8e..0000000
16--- a/daily-package-testing/README
17+++ /dev/null
18@@ -1,18 +0,0 @@
19-About
20-=====
21-
22-This Vagrantfile can be used to see how packaged version of plainbox from the
23-ppa looks like.
24-
25-The package is built with this recipe:
26-https://code.launchpad.net/~checkbox-dev/+recipe/plainbox-daily
27-
28-Usage instructions
29-==================
30-
31-$TARGET is either precise or quantal
32-
33-$ vagrant up $TARGET
34-$ vagrant ssh $TARGET
35-$ plainbox --help
36-$ exit
37diff --git a/daily-package-testing/Vagrantfile b/daily-package-testing/Vagrantfile
38deleted file mode 100644
39index 28213a8..0000000
40--- a/daily-package-testing/Vagrantfile
41+++ /dev/null
42@@ -1,48 +0,0 @@
43-# -*- mode: ruby -*-
44-# vi: set ft=ruby sw=2 ts=2 :
45-
46-Vagrant::Config.run do |config|
47-
48- config.ssh.timeout = 60
49-
50- # Define a Ubuntu Server image (cloud) for the 12.04 release (precise)
51- config.vm.define :precise do |precise_config|
52- precise_config.vm.box = "precise-cloud-i386"
53- precise_config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-i386-vagrant-disk1.box"
54- end
55-
56- # Define a Ubuntu Server image (cloud) for the 12.10 release (quantal)
57- config.vm.define :quantal do |quantal_config|
58- quantal_config.vm.box = "quantal-cloud-i386"
59- quantal_config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/quantal/current/quantal-server-cloudimg-i386-vagrant-disk1.box"
60- end
61-
62- # Define a Ubuntu Server image (cloud) for the 13.04 release (raring)
63- config.vm.define :raring do |raring_config|
64- raring_config.vm.box = "raring-cloud-i386"
65- raring_config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/raring/current/raring-server-cloudimg-i386-vagrant-disk1.box"
66- end
67-
68- # For debugging and later future GUI testing
69- if ENV.key? "VAGRANT_GUI"
70- config.vm.boot_mode = :gui
71- end
72-
73- # Setup an apt cache if one is available
74- if ENV.key? "VAGRANT_APT_CACHE"
75- config.vm.provision :shell, :inline => "echo 'Acquire::http { Proxy \"#{ENV['VAGRANT_APT_CACHE']}\"; };' > /etc/apt/apt.conf"
76- end
77-
78- # Update to have the latest packages, this is needed because the image comes
79- # with an old (and no longer working) apt cache and links to many packages no
80- # longer work.
81- config.vm.provision :shell, :inline => "apt-get update && DEBIAN_FRONTEND=noninteractive apt-get dist-upgrade --yes"
82- # Install package that provides add-apt-repository
83- config.vm.provision :shell, :inline => "apt-get install --yes python-software-properties"
84- # Add the checkbox-dev/ppa ppa with daily builds
85- config.vm.provision :shell, :inline => "add-apt-repository ppa:checkbox-dev/ppa"
86- # Update apt cache again
87- config.vm.provision :shell, :inline => "apt-get update"
88- # Install plainbox
89- config.vm.provision :shell, :inline => "apt-get install --yes plainbox"
90-end
91diff --git a/daily-package-testing/test-in-vagrant.sh b/daily-package-testing/test-in-vagrant.sh
92deleted file mode 100755
93index c21b00f..0000000
94--- a/daily-package-testing/test-in-vagrant.sh
95+++ /dev/null
96@@ -1,59 +0,0 @@
97-#!/bin/sh
98-# Runs simple smoke tests on the current package in the daily ppa.
99-
100-mkdir -p vagrant-logs
101-
102-test -z $(which vagrant) && echo "You need to install vagrant first" && exit
103-
104-outcome=0
105-# XXX: this list needs to be in sync with plainbox/daily-pkg-testing/Vagrantfile
106-target_list="precise quantal raring"
107-for target in $target_list; do
108- # Bring up target if needed
109- if ! vagrant status $target | grep -q running; then
110- echo "[$target] Bringing VM 'up'"
111- if ! vagrant up $target >vagrant-logs/$target.startup.log 2>vagrant-logs/$target.startup.err; then
112- outcome=1
113- echo "[$target] Unable to 'up' VM!"
114- echo "[$target] stdout: $(pastebinit vagrant-logs/$target.startup.log)"
115- echo "[$target] stderr: $(pastebinit vagrant-logs/$target.startup.err)"
116- echo "[$target] NOTE: unable to execute tests, marked as failed"
117- continue
118- fi
119- fi
120- # Display something before the first test output
121- echo "[$target] Starting tests..."
122- # Test that plainbox --help works correctly
123- if vagrant ssh $target -c 'plainbox --help' >vagrant-logs/$target.help.log 2>vagrant-logs/$target.help.err; then
124- echo "[$target] packaged PlainBox plainbox --help: pass"
125- else
126- outcome=1
127- echo "[$target] packaged PlainBox plainbox --help: fail"
128- echo "[$target] stdout: $(pastebinit vagrant-logs/$target.help.log)"
129- echo "[$target] stderr: $(pastebinit vagrant-logs/$target.help.err)"
130- fi
131- case $VAGRANT_DONE_ACTION in
132- suspend)
133- # Suspend the target to conserve resources
134- echo "[$target] Suspending VM"
135- if ! vagrant suspend $target >vagrant-logs/$target.suspend.log 2>vagrant-logs/$target.suspend.err; then
136- echo "[$target] Unable to suspend VM!"
137- echo "[$target] stdout: $(pastebinit vagrant-logs/$target.suspend.log)"
138- echo "[$target] stderr: $(pastebinit vagrant-logs/$target.suspend.err)"
139- echo "[$target] You may need to manually 'vagrant destroy $target' to fix this"
140- fi
141- ;;
142- destroy)
143- # Destroy the target to work around virtualbox hostsf bug
144- echo "[$target] Destroying VM"
145- if ! vagrant destroy --force $target >vagrant-logs/$target.destroy.log 2>vagrant-logs/$target.destroy.err; then
146- echo "[$target] Unable to destroy VM!"
147- echo "[$target] stdout: $(pastebinit vagrant-logs/$target.suspend.log)"
148- echo "[$target] stderr: $(pastebinit vagrant-logs/$target.suspend.err)"
149- echo "[$target] You may need to manually 'vagrant destroy $target' to fix this"
150- fi
151- ;;
152- esac
153-done
154-# Propagate failure code outside
155-exit $outcome
156diff --git a/docs/author/index.rst b/docs/author/index.rst
157index 8e213ae..b22bb46 100644
158--- a/docs/author/index.rst
159+++ b/docs/author/index.rst
160@@ -8,10 +8,8 @@ core.
161
162 .. toctree::
163 intro.rst
164- tutorial.rst
165 qml-job-tutorial.rst
166 providers.rst
167- whitelists.rst
168 rfc822.rst
169 faq.rst
170
171diff --git a/docs/author/intro.rst b/docs/author/intro.rst
172index 2bbabb7..2dae15d 100644
173--- a/docs/author/intro.rst
174+++ b/docs/author/intro.rst
175@@ -49,7 +49,7 @@ Terminology
176 In developing or using Plainbox, you'll run into several unfamiliar terms.
177 Check the :doc:`../glossary` to learn what they mean. In fact, you should
178 probably check it now. Pay particular attention to the terms *Checkbox*,
179-*Plainbox*, *job*, *provider*, and *whitelist*.
180+*Plainbox*, *job* and *provider*.
181
182 Getting Started
183 ---------------
184@@ -73,7 +73,7 @@ tests. First up is a welcome screen:
185 select tests.
186
187 When you press the Enter key, ``checkbox-cli`` lets you select which
188-whitelist to use:
189+test plan to use:
190
191 .. image:: cc2.png
192 :height: 343
193@@ -81,7 +81,7 @@ whitelist to use:
194 :scale: 100
195 :alt: checkbox-cli enables you to select which test suite to run.
196
197-With a whitelist selected, you can choose the individual tests to run:
198+With a test plan selected, you can choose the individual tests to run:
199
200 .. image:: cc3.png
201 :height: 600
202@@ -117,7 +117,7 @@ A provider is described in a configuration file (stored in
203 ``/usr/share/plainbox-providers-1``). This file describes where to find all
204 the files from the provider. This file is usually managed automatically
205 (more on this later). A provider can ship jobs, binaries, data and
206-whitelists.
207+test plans.
208
209 A **job** or **test** is the smallest unit or description that Plainbox
210 knows about. It describes a single test (historically they're called
211@@ -140,10 +140,6 @@ types include (but are not limited to):
212 * ``shell`` -- An automated test that requires no user interaction; the
213 test is passed or failed on the basis of the return value of the script
214 or command.
215- * ``local`` -- This type of job is similar to a ``shell`` test, but it
216- supports creating multiple tests from a single definition (say, to test
217- all the Ethernet ports on a computer). Jobs using the ``local`` plugin
218- are run when Plainbox is initialized.
219 * ``user-interact`` -- A test that asks the user to perform some action
220 *before* the test is performed. The test then passes or fails
221 automatically based on the output of the test. An example is
222@@ -162,170 +158,9 @@ types include (but are not limited to):
223 which probes the system to determine the maximum supported resolution
224 and then asks the user to confirm that the resolution is correct.
225
226-A fairly complex example definition is::
227-
228- plugin: local
229- _summary: Automated test to walk multiple network cards and test each one in sequence.
230- id: ethernet/multi_nic
231- requires:
232- device.category == 'NETWORK'
233- _description: Automated test to walk multiple network cards and test each one in sequence.
234- command:
235- cat <<'EOF' | run_templates -s 'udev_resource | filter_templates -w "category=NETWORK" | awk "/path: / { print \$2 }" | xargs -n 1 sh -c "for i in \``ls /sys\$0/net 2>/dev/null\``; do echo \$0 \$i; done"'
236- plugin: shell
237- id: ethernet/multi_nic_$2
238- requires:
239- package.name == 'ethtool'
240- package.name == 'nmap'
241- device.path == "$1"
242- user: root
243- environ: TEST_TARGET_FTP TEST_TARGET_IPERF TEST_USER TEST_PASS
244- command: network test -i $2 -t iperf --fail-threshold 80
245- estimated_duration: 330.0
246- description:
247- Testing for NIC $2
248- EOF
249-
250-Key points to note include:
251-
252- * If a field name begins with an underscore, its value can be localized.
253- * The values of fields can appear on the same line as their field names,
254- as in ``plugin: local``; or they can appear on a subsequent line, which
255- is indented, as in the preceding example's ``requires: device.category
256- == 'NETWORK'``.
257- * The ``requires`` field can be used to specify dependencies; if the
258- specified condition is not met, the test does not run.
259- * The ``command`` field specifies the command that's used to run the test.
260- This can be a standard Linux command (or even a set of commands) or a
261- Checkbox test script. In this example's ``local`` test definition, the
262- first ``command`` line generates a list of network devices that is fed
263- to an embedded test, which is defined beginning with the second
264- ``plugin`` line immediately following the first ``command`` line.
265- * In this example, the line that reads ``EOF`` ends the
266- ``ethernet/ethtool_multi_nic_$2`` test's command; it's matched to the
267- ``EOF`` that's part of ``cat << 'EOF'`` near the start of that command.
268-
269 Each provider has a ``bin`` directory and all binaries there are available
270 in the path.
271
272-Whitelists
273-``````````
274-
275-In the job files we have a "universe" of known jobs. We don't normally want
276-to run them all; rather we want to select a subset depending on what we're
277-testing, and maybe give the user a way to fine-tune that selection. Also,
278-we need a way to determine the order in which they will run, beyond what
279-dependencies may provide. This is where the whitelist comes in; think of it
280-as a mask or selection filter from the universe of jobs. Whitelists support
281-regular expressions, and Plainbox will attempt to run tests in the order
282-shown in the whitelist. Again, providers ship whitelists in a specific
283-directory, and you can use ``plainbox`` to run a specific whitelist with
284-the ``-w`` option.
285-
286-You can also use ``plainbox`` to run a test with the ``-i`` syntax. This is
287-good for quickly running a job and ensuring it works well.
288-
289-Let's look at ``checkbox-cli`` for a moment. This is a "launcher"; it
290-specifies a set of configuration options for a specific testing purpose.
291-This enables us to create mini-clients for each testing purpose, without
292-changing the core utility (``checkbox-launcher``). For instance, let's look
293-at the launcher for ``canonical-certification-server``, which appears in
294-``./providers/plainbox-provider-certification-server/launcher/canonical-certification-server``
295-in the Checkbox source tree::
296-
297- #!/usr/bin/env checkbox-launcher
298- [welcome]
299- text = Welcome to System Certification!
300- This application will gather information from your system. Then you will be
301- asked manual tests to confirm that the system is working properly. Finally,
302- you will be asked for the Secure ID of the computer to submit the
303- information to the certification.canonical.com database.
304- To learn how to create or locate the Secure ID, please see here:
305- https://certification.canonical.com/
306-
307- [suite]
308- # Whitelist(s) displayed in the suite selection screen
309- whitelist_filter = ^((network|storage|usb|virtualization)-only)|(server-(full|functional)-14.04)$
310- # Whitelist(s) pre-selected in the suite selection screen, default whitelist(s)
311- whitelist_selection = ^server-full-14.04$
312-
313- [transport]
314- submit_to = certification
315-
316- [config]
317- config_filename = canonical-certification.conf
318-
319-A launcher such as this sets up an environment that includes introductory
320-text to be shown to users, a filter to determine what whitelists to present
321-as options, information on where to (optionally) submit results, and a
322-configuration filename. This allows each provider to ship a launcher or
323-binary with which to launch its relevant tests.
324-
325-Developing Tests
326-````````````````
327-
328-One way to deliver tests via Plainbox is to start your own provider. To
329-learn how to do that, see the :ref:`tutorial`.
330-
331-In other cases you want to add tests to the main Checkbox repository (which
332-is also what we recommend to keep tests centralized, unless they're so
333-purpose-specific that this makes no sense).
334-
335-This is a bit easier because the provider in question already exists. So
336-let's get started by branching a copy of ``lp:checkbox``. In brief, you
337-should change to your software development directory and type ``bzr branch
338-lp:checkbox my-branch`` to create a copy of the ``checkbox`` Launchpad
339-project in the ``my-branch`` subdirectory. You can then edit the files in
340-that subdirectory, upload the results to your own Launchpad account, and
341-request a merge.
342-
343-To begin, consider the files and subdirectories in the main Checkbox
344-development directory (``my-branch`` if you used the preceding ``bzr``
345-command without change):
346-
347- * ``checkbox-gui`` -- Checkbox GUI components, used in desktop/laptop
348- testing
349- * ``checkbox-ng`` -- The Plainbox-based version of Checkbox
350- * ``checkbox-support`` -- Support code for many providers
351- * ``checkbox-touch`` -- A Checkbox frontend optimized for touch/tablet
352- devices
353- * ``mk-venv`` -- A symbolic link to a script used to set up an environment
354- for testing Checkbox
355- * ``plainbox`` -- A Python3 library and development tools at the heart of
356- Plainbox
357- * ``plainbox-client`` -- Unfinished Python3 interface for Checkbox
358- * ``providers`` -- Provider definitions, including test scripts
359- * ``README.md`` -- A file describing the contents of the subdirectory in
360- greater detail
361- * ``setup.py`` -- A setup script
362- * ``support`` -- Support code that's not released
363- * ``tarmac-verify`` -- A support script
364- * ``test-in-lxc.sh`` -- A support script for testing in an LXC
365- * ``test-in-vagrant.sh`` -- A support script for testing with Vagrant
366- * ``test-with-coverage`` -- A link to a support script for testing with
367- coverage
368- * ``Vagrantfile`` -- A Vagrant configuration file
369-
370-Let's say I want to write a test to ensure that the ubuntu user exists in
371-``/etc/passwd``. You need to remove any existing Checkbox provider
372-packages, lest they interfere with your new or modified tests. The
373-``setup.py`` script will set up a Plainbox development environment for you.
374-
375-We can write a simple job here, then add a requirement, perhaps a
376-dependency, then a script in the directory. Note that scripts can be
377-anything that's executable, we usually prefer either shell or Python but
378-anything goes.
379-
380-Plainbox will supply two environment variables, ``PLAINBOX_PROVIDER_DATA``
381-and ``SHARE``, we usually try to use them in the job description only, not
382-in the scripts, to keep the scripts Plainbox-agnostic if possible.
383-
384-Once the test is running correctly, we can create a whitelist with a few
385-tests and name it.
386-
387-Once we get everything running correctly we can prepare and propose a merge
388-request using ``bzr`` as usual.
389-
390 Other Questions
391 ---------------
392
393diff --git a/docs/author/provider-files.rst b/docs/author/provider-files.rst
394index 7a33346..34f21d4 100644
395--- a/docs/author/provider-files.rst
396+++ b/docs/author/provider-files.rst
397@@ -53,11 +53,6 @@ jobs_dir
398 Absolute pathname to a directory with :term:`job definitions <job>`
399 as individual ``.txt`` files using the :doc:`job file format <jobs>`.
400
401-whitelists_dir
402- Absolute pathname to a directory with :term:`whitelists <whitelist>`
403- as individual ``.whitelist`` files using the
404- :doc:`whitelist format <whitelists>`.
405-
406 bin_dir
407 Absolute pathname to a directory with additional executables required by
408 any of the job definitions.
409@@ -80,7 +75,6 @@ location
410 Variable Default Value
411 ================ =====================
412 jobs_dir $location/jobs
413- whitelists_dir $location/whitelists
414 bin_dir $location/bin
415 data_dir $location/data
416 locale_dir $location/locale
417diff --git a/docs/author/provider-namespaces.rst b/docs/author/provider-namespaces.rst
418index 4459e50..e25036b 100644
419--- a/docs/author/provider-namespaces.rst
420+++ b/docs/author/provider-namespaces.rst
421@@ -120,7 +120,7 @@ The part of the provide name before the colon is used as the name-space. The
422 colon is *not* a part of the name-space.
423
424 The implicit name-space is used to construct non-partial job definition names
425-as well as to implicitly prefix each pattern inside :term:`whitelists <whitelist>`.
426+as well as to implicitly prefix each pattern inside test plans.
427
428 Using Explicit Name-Spaces
429 --------------------------
430@@ -133,15 +133,15 @@ Explicit name-spaces need to be used in two situations:
431 This is required as any partial ID may silently change the job it resolves
432 to and we didn't want to introduce that ambiguity.
433
434-2. When including a job from another name-space inside a whitelist, e.g.::
435+2. When including a job from another name-space inside a test plan, e.g.::
436
437- ~/com.example.some:provider$ cat whitelists/cross.whitelist
438+ ~/com.example.some:provider$ cat units/test-plan.pxu
439 job-a
440 job-b
441 com\.example\.other::job-a
442 ~com.example.some:provider$
443
444- Here the whitelist names three jobs:
445+ Here the test plan names three jobs:
446
447 * com.example.some::job-a
448 * com.example.some::job-b
449diff --git a/docs/author/provider-template.rst b/docs/author/provider-template.rst
450index 91a1db3..ae5bb76 100644
451--- a/docs/author/provider-template.rst
452+++ b/docs/author/provider-template.rst
453@@ -23,17 +23,14 @@ The following files and directories are generated::
454 ├── data
455 │   ├── example.dat
456 │   └── README.md
457- ├── jobs
458- │   ├── examples-intermediate.txt
459- │   ├── examples-normal.txt
460- │   └── examples-trivial.txt
461 ├── manage.py
462 ├── po
463 │   └── POTFILES.in
464 ├── README.md
465- └── whitelists
466- ├── normal.whitelist
467- └── trivial.whitelist
468+ └── units
469+    ├── examples-intermediate.txt
470+    ├── examples-normal.txt
471+    └── examples-trivial.txt
472
473 Generated Content
474 =================
475@@ -59,10 +56,8 @@ README.md
476 Plainbox parlance, is the smallest piece of executable test code. Each
477 job has a name and a number of other attributes.
478
479- Jobs can be arranged in lists, test plans if you will that are known
480- as "whitelists". Those are defined in the ``whitelists/`` directory,
481- this time one per file. You can create as many whitelists as you need,
482- referring to arbitrary subsets of your jobs.
483+ Jobs can be arranged in lists, test plans if you will. You can create as
484+ many test plans as you need, referring to arbitrary subsets of your jobs.
485
486 Then there are the ``bin/`` and ``data/`` directories. Those are
487 entirely for custom content you may need. You can put arbitrary
488@@ -340,7 +335,7 @@ jobs/examples-intermediate.txt
489 estimated_duration: 30
490
491
492-po/PORFILES.in
493+po/POTFILES.in
494 --------------
495
496 ::
497@@ -350,20 +345,3 @@ po/PORFILES.in
498 [type: gettext/rfc822deb] jobs/examples-normal.txt
499 [type: gettext/rfc822deb] jobs/examples-intermediate.txt
500 manage.py
501-
502-whitelists/trivial.whitelist
503-----------------------------
504-
505-::
506-
507- # select two trivial jobs by directly selecting their names
508- examples/trivial/always-pass
509- examples/trivial/always-fail
510-
511-whitelists/normal.whitelist
512----------------------------
513-
514-::
515-
516- # use regular expression to select all normal jobs
517- examples/normal/.*
518diff --git a/docs/author/providers.rst b/docs/author/providers.rst
519index 6640750..cc1bb6c 100644
520--- a/docs/author/providers.rst
521+++ b/docs/author/providers.rst
522@@ -5,7 +5,7 @@ Providers
523 Providers are a new feature introduced in Plainbox 0.5. They allow third party
524 developers to produce and maintain private and public test collections.
525
526-All :term:`jobs <job>` and :term:`whitelists <whitelist>` are now loaded from a provider. This
527+All :term:`jobs <job>` and test plans are now loaded from a provider. This
528 also affects the :term:`Checkbox` project that now produces a custom user
529 interface and a number of providers for various purposes.
530
531diff --git a/docs/author/tutorial.rst b/docs/author/tutorial.rst
532deleted file mode 100644
533index 892cb19..0000000
534--- a/docs/author/tutorial.rst
535+++ /dev/null
536@@ -1,177 +0,0 @@
537-.. _tutorial:
538-
539-========
540-Tutorial
541-========
542-
543-To best illustrate how providers work, we will walk through creating one
544-step-by-step. At the end of this tutorial you will have a provider which adds
545-a new :term:`whitelist`, several new jobs and the scripts and test data
546-supporting those jobs. Before starting this tutorial you will need to have a
547-running version of :term:`Plainbox` installed. You can either install it from
548-the repositories of Debian or its derivatives by running ``apt-get install
549-plainbox``, or if you prefer to work with the source, see :doc:`Getting
550-started with development <../dev/intro>`. There is also a Launchpad PPA with
551-the very latest development build for Ubuntu, which is `ppa:checkbox-dev/ppa`.
552-
553-#. To get started we create an initial template for our provider by running
554- ``plainbox startprovider com.example:myprovider``.
555-
556-#. This will create a directory called ``com.example:myprovider``.
557- Change to this directory and you will see that it contains::
558-
559- /bin
560- /data
561- /integration-tests
562- /jobs
563- manage.py
564- README.md
565- /whitelists
566-
567- The ``manage.py`` script is a helper script for developing the provider.
568- It provides a set of commands which assist in validating the correctness
569- of the provider and making it ready for distribution.
570-
571-#. Let’s create some jobs first by changing to the jobs directory. It currently
572- contains a file called category.txt which serves as an example of how
573- jobs should look. Let’s delete it and instead create a file called
574- ``myjobs.txt``. This can contain the following simple jobs::
575-
576- plugin: shell
577- name: myjobs/shell_command
578- command: true
579- _description:
580- An example job that uses a command provided by the shell.
581-
582- plugin: shell
583- name: myjobs/provider_command
584- command: mycommand
585- _description:
586- An example job that uses a test command provided by this provider.
587-
588- At this point we can check that everything looks okay by running the command
589- ``./manage.py info`` which displays some information about the provider. The
590- output should be something like::
591-
592- [Provider MetaData]
593- name: com.example:myprovider
594- version: 1.0
595- [Job Definitions]
596- 'myjobs/builtin_command', from jobs/myjobs.txt:1-5
597- 'myjobs/provider_command', from jobs/myjobs.txt:7-11
598- [White Lists]
599- 'category', from whitelists/category.whitelist:1-1
600-
601- This shows all three jobs from the job file we added - great!
602-
603-#. Next we need to change directory to ``bin`` to add the command used by the
604- job ``myjobs/this_provider_command``. We create a file there called
605- ``mycommand`` which contains the following text::
606-
607- #!/bin/sh
608- test `cat $CHECKBOX_SHARE/data/testfile` = 'expected'
609-
610- This needs to be executable to be used in the job command so we need to run
611- ``chmod a+x mycommand`` to make it executable.
612-
613- You'll notice the command uses a file in ``$CHECKBOX_SHARE/data`` - we'll
614- add this file to our provider next.
615-
616-#. Because the command we’re using uses a file that we expect to be located in
617- ``$CHECKBOX_SHARE/data``, we need to add this file to our provider so that
618- after the provider is installed this file is available in that location.
619- First we need to change to the directory called ``data``, then as indicated
620- by the contents of the script we wrote in the previous step, we need to
621- create a file there called ``testfile`` with the contents::
622-
623- expected
624-
625- As simple as that!
626-
627-#. Lastly we need to add a :term:`whitelist` that utilizes the jobs we created
628- earlier. We need to change to the directory called ``whitelists``. As with
629- the ``jobs`` directory there is already an example file there called
630- ``category.whitelist``. We can delete that and add a file called
631- ``mywhitelist.whitelist``. The contents should be::
632-
633- myjobs/shell_command
634- myjobs/provider_command
635-
636- The ``miscellanea/submission_resources`` and ``graphics/glxgears`` jobs
637- are from the default provider that is part of Plainbox.
638-
639- We can check that everything is correct with the whitelist by running the
640- ``./manage.py info`` command again. The output should be like::
641-
642- [Provider MetaData]
643- name: com.example:myprovider
644- version: 1.0
645- [Job Definitions]
646- 'myjobs/builtin_command', from jobs/myjobs.txt:1-5
647- 'myjobs/provider_command', from jobs/myjobs.txt:7-11
648- [White Lists]
649- 'mywhitelist', from whitelists/mywhitelist.whitelist:1-2
650-
651- Our new :term:`whitelist` is listed there.
652-
653-#. Now we have a provider we need to test it to make sure everything is
654- correct. The first thing to do is to install the provider so that it
655- it visible to Plainbox. Run ``./manage.py develop`` then run
656- ``plainbox dev list provider``. Your provider should be in the list
657- that is displayed.
658-
659-#. We should also make sure the whole provider works end-to-end by running
660- the :term:`whitelist` which it provides. Run the following command -
661- ``plainbox run -w whitelists/mywhitelist.whitelist``.
662-
663-#. Assuming everything works okay, we can now package the provider for
664- distribution. This involves creating a basic ``debian`` directory
665- containing all of the files needed for packaging your provider. Create
666- a directory called ``debian`` at the base of your provider, and then
667- create the following files within it.
668-
669- ``compat``::
670-
671- 9
672-
673- ``control``::
674-
675- Source: plainbox-myprovider
676- Section: utils
677- Priority: optional
678- Maintainer: Brendan Donegan <brendan.donegan@canonical.com>
679- Standards-Version: 3.9.3
680- X-Python3-Version: >= 3.2
681- Build-Depends: debhelper (>= 9.2),
682- lsb-release,
683- python3 (>= 3.2),
684- python3-plainbox
685-
686- Package: plainbox-myprovider
687- Architecture: all
688- Depends: plainbox-provider-checkbox
689- Description: My whitelist provider
690- A provider for Plainbox.
691-
692- ``rules``::
693-
694- #!/usr/bin/make -f
695- %:
696- dh "$@"
697-
698- override_dh_auto_build:
699- $(CURDIR)/manage.py install
700-
701- Note that the ``rules`` file must be executable. Make it so with
702- ``chmod a+x rules``. Also, be careful with the indentation in the
703- file - all indents must be actual TAB characters, not four spaces
704- for example.
705-
706- ``source/format``::
707-
708- 3.0 (native)
709-
710- Finally we should create a ``changelog`` file. The easiest way to do this
711- is to run the command ``dch --create 'Initial release.'``. You'll need to
712- edit the field ``PACKAGE`` to the name of your provider and the field
713- ``VERSION`` to something like ``0.1``.
714diff --git a/docs/author/whitelists.rst b/docs/author/whitelists.rst
715deleted file mode 100644
716index 7b5f162..0000000
717--- a/docs/author/whitelists.rst
718+++ /dev/null
719@@ -1,120 +0,0 @@
720-========================
721-Checkbox Whitelist Files
722-========================
723-
724-When creating a test suite for a particular purpose, it will be necessary to
725-specify which tests to run and which order they should run in. For this purpose
726-Checkbox provides the concept of Whitelists.
727-
728-Whitelist Format
729-================
730-
731-A whitelist is a text file containing a line-seperated sequence of patterns,
732-each representing one or more 'jobs'. These patterns are in the Python regular
733-expression syntax. Comments may be included in the file by starting the line
734-with '#'.
735-
736-Minimal Whitelist File
737-======================
738-
739-In order to be useful a whitelist file needs to include a particular subset of
740-jobs which provide Checkbox with all of the information it needs to run tests
741-properly. These include jobs which attach hardware information and resource
742-jobs which provide other jobs with information of the environment they a
743-re running in (available hardware, available packages etc). To make this easy
744-to do a single job exists whose purpose is to execute all of these other jobs::
745-
746- miscellanea/submission-resources
747-
748-This should be included as the first job in any whitelist.
749-
750-Job Categories
751-==============
752-
753-In order to allow Checkbox to display jobs by category in the UI it is
754-necessary to include a particular local job which itself generates jobs which
755-belong to that category. This job will normally look like ``__<category>__``
756-where <category> is the name of the job file which contains the job. This is
757-indicated again by the prefix of the job (before the ``/`` in the job name).
758-As a quick example, the job ``graphics/glxgears`` is contained in
759-``graphics.txt``. Therefore we should include the ``__graphics__`` job so that
760-the ``graphics/glxgears`` job shows correctly under the category. The
761-``__graphics__`` job itself looks like::
762-
763- name: __graphics__
764- plugin: local
765- _description: Graphics tests
766- command:
767- shopt -s extglob
768- cat $CHECKBOX_SHARE/jobs/graphics.txt?(.in)
769-
770-Checkbox will interpret this job as a request to display any job in
771-``graphics.txt`` (or its untranslated version ``graphics.txt.in``) under the
772-heading shown in the description of this job (in this case 'Graphics tests').
773-
774-Tutorial
775-========
776-
777-To compound what we discussed before, below is a brief tutorial which walks
778-through assembling a basic whitelist file.
779-
780-1. First we need to create a file, let's name it tutorial.whitelist.
781-Whitelists don't have to end with the .whitelist suffix but this is the
782-convention used to help identify them.
783-2. We start by adding the one job that is required for all whitelists, as
784-explained above in the section 'Minimal Whitelist File', so our whitelist file
785-looks like::
786-
787- miscellanea/submission-resources
788-
789-3. Next we should choose some jobs that we want to run. This all depends on
790-your specific use-case of course, but I've selected a few jobs that will help
791-clearly illustrate more of the concepts involved in whitelists. These jobs will
792-give us a whitelist file that looks like::
793-
794- miscellanea/submission-resources
795- cpu/clocktest
796- ethernet/multi_nic
797- ethernet/multi_nic_eth0
798- graphics/glxgears
799-
800- If we run this whitelist now then all of these jobs will be executed and a
801- valid test submission will be created, but we can still improve it in a couple
802- of ways.
803-
804-4. The first way is by adding the necessary jobs to allow the Checkbox UI to
805-group the jobs into specific categories. To do this we need to add a job with
806-a name like ``__<category>__`` for each category. We have three categories in
807-our whitelist file - cpu, ethernet and graphics. The category of the job is
808-the prefix of the job name prior to the ``/``. So now our whitelist file looks
809-like::
810-
811- miscellanea/submission-resources
812- __cpu__
813- __ethernet__
814- __graphics__
815- cpu/clocktest
816- ethernet/multi_nic
817- ethernet/multi_nic_eth0
818- graphics/glxgears
819-
820- Now the Checkbox UI will group the jobs into these categories.
821-
822-5. Although it's not immediately apparent there is another problem with this
823-whitelist. The ``ethernet/multi_nic`` tests are only able to include one job
824-for the ethernet port 'eth0'. It would be better if we included all of the
825-jobs generated by 'ethernet/multi_nic', no matter how many ethernet ports are
826-present on the system under test. The best way to do this is to write the
827-pattern so that it matches all of the possible job names. We can take advantage
828-of the Python regular expression syntax and use the ``\d`` special character
829-to match any decimal number. After doing this the whitelist file will look
830-like this::
831-
832- miscellanea/submission-resources
833- __cpu__
834- __ethernet__
835- __graphics__
836- cpu/clocktest
837- ethernet/multi_nic
838- ethernet/multi_nic_eth\d
839- graphics/glxgears
840diff --git a/docs/dev/old.rst b/docs/dev/old.rst
841index 151f21c..7921f68 100644
842--- a/docs/dev/old.rst
843+++ b/docs/dev/old.rst
844@@ -144,8 +144,6 @@ dependencies) being added to the repository.
845 Implementation issues
846 ---------------------
847
848-There are two issues that are known at this time:
849-
850 * There is too much checkbox-specific knowledge which really belongs
851 elsewhere. We are working to remove that so that non-checkbox jobs
852 can be introduced later. There is a branch in progress that entirely
853@@ -155,15 +153,6 @@ There are two issues that are known at this time:
854 that was previously internal (most notably a way to add new jobs and
855 resources).
856
857-* The way jobs are currently selected is unfortunate because of local jobs
858- that can add new jobs to the system. This causes considerable complexity
859- at the application level where the application must check if each
860- executed job is a 'local' job and re-compute the desired_job_list. This
861- should be replaced by a matcher function that can be passed to
862- SessionState once so that desired_job_list is re-evaluated internally
863- whenever job_list changes.
864-
865-
866 :class:`~plainbox.impl.job.JobDefinition`
867 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
868
869@@ -211,49 +200,6 @@ plugin == "manual"
870 This value is used for fully manual jobs. It has no special handling in the core
871 apart from requiring a human-provided outcome (pass/fail classification)
872
873-.. _local:
874-
875-plugin == "local"
876-#################
877-
878-This value is used for special job generator jobs. The output of such jobs is
879-interpreted as additional jobs and is identical in effect to loading such jobs
880-from a job definition file.
881-
882-There are two practical uses for such jobs:
883-
884-* Some local jobs are used to generate a number of jobs for each object.
885- This is needed where the tested machine may have a number of such objects
886- and each requires unique testing. A good example is a computer where all
887- network tests are explicitly "instantiated" for each network card
888- present.
889-
890- This is a valid use case but is rather unfortunate for architecture of
891- Plainbox and there is a desire to replace it with equally-expressive
892- pattern jobs. The advantage is that unlike local jobs (which cannot be
893- "discovered" without enduring any potential side effects that may be
894- caused by the job script command) pattern jobs would allow the core to
895- determine the names of jobs that can be generated and, for example,
896- automatically determine that a pattern job needs to be executed as a
897- dependency of a phantom (yet undetermined) job with a given name.
898-
899- The solution with "pattern" jobs may be executed in future phases of
900- Plainbox development. Currently there is no support for that at all.
901-
902- Currently Plainbox cannot determine job dependencies across local jobs.
903- That is, unless a local job is explicitly requested (in the desired job
904- list) Plainbox will not be able to run a job that is generated by a local
905- job at all and will treat it as if that job never existed.
906-
907-* Some local jobs are used to create a form of informal "category".
908- Typically all such jobs have a leading and trailing double underscore,
909- for example '__audio__'. This is currently being used by Checkbox for
910- building a hierarchical tree of tests that the user may select.
911-
912- Since this has the same flaws as described above (for pattern jobs) it
913- will likely be replaced by an explicit category field that can be
914- specified each job.
915-
916 plugin == "resource"
917 ####################
918
919diff --git a/docs/dev/trusted-launcher.rst b/docs/dev/trusted-launcher.rst
920index d74735e..a0cc565 100644
921--- a/docs/dev/trusted-launcher.rst
922+++ b/docs/dev/trusted-launcher.rst
923@@ -145,7 +145,7 @@ Usage
924 .. code-block:: text
925
926 plainbox-trusted-launcher-1 [-h] (--hash HASH | --warmup)
927- [--via LOCAL-JOB-HASH]
928+ [--via GENERATOR-JOB-HASH]
929 [NAME=VALUE [NAME=VALUE ...]]
930
931 positional arguments:
932@@ -156,7 +156,7 @@ Usage
933 --hash HASH job hash to match
934 --warmup Return immediately, only useful when used with
935 pkexec(1)
936- --via LOCAL-JOB-HASH Local job hash to use to match the generated job
937+ --via GENERATOR-JOB-HASH Generator job hash to use to match the generated job
938
939 .. note::
940
941@@ -185,15 +185,11 @@ thanks to the installed policy file the authentication will be kept.
942 Special case of jobs using the Checkbox local plugin
943 ----------------------------------------------------
944
945-For jobs generated from :ref:`local <local>` jobs (e.g.
946+For jobs generated from resources jobs (e.g.
947 disk/read_performance.*) the trusted launcher is started with ``--via`` meaning
948-that we have to first eval a local job to find a hash match. Once a match is
949+that we have to first eval a generator job to find a hash match. Once a match is
950 found, the job command is executed.
951
952 .. code-block:: bash
953
954- $ pkexec plainbox-trusted-launcher-1 --hash JOB-HASH --via LOCAL-JOB-HASH
955-
956-.. note::
957-
958- it will obviously fail if any local job can ever generate another local job.
959+ $ pkexec plainbox-trusted-launcher-1 --hash JOB-HASH --via GENERATOR-JOB-HASH
960diff --git a/docs/glossary.rst b/docs/glossary.rst
961index 6174025..212d9a2 100644
962--- a/docs/glossary.rst
963+++ b/docs/glossary.rst
964@@ -46,12 +46,11 @@ Glossary
965 necessary for end-user work. ``plainbox`` is usually installed
966 explicitly if needed.
967
968- whitelist
969+ test plan
970
971- Whitelists are text files used by Checkbox to select jobs for
972- execution. They can include simple regular expressions to match and
973- pick many similar jobs at once. For more information see
974- :doc:`Checkbox Whitelist Files <author/whitelists>`
975+ Test plans are text files used by Checkbox to select jobs for
976+ execution. They can include simple regular expressions to match and
977+ pick many similar jobs at once.
978
979 job
980
981@@ -63,7 +62,7 @@ Glossary
982
983 provider
984
985- A container for jobs, whitelists, private executables and data.
986+ A container for jobs, test plans, private executables and data.
987 Providers are the foundation of Plainbox as they *provide* all of the
988 content. Providers can be created and managed by any entity, separately
989 from the Checkbox project.
990diff --git a/docs/manpages/plainbox-dev-analyze.rst b/docs/manpages/plainbox-dev-analyze.rst
991index e5fe5dc..23fc999 100644
992--- a/docs/manpages/plainbox-dev-analyze.rst
993+++ b/docs/manpages/plainbox-dev-analyze.rst
994@@ -15,13 +15,6 @@ plainbox-dev-analyze (1)
995 and the command prints nothing at all) to inspect certain aspects of the
996 hypothetical session
997
998- The only exception to the rule above is the ``--run-local`` option. With that
999- option all local jobs and their dependencies *are* started. This is
1000- technically required to correctly emulate the behavior of ``plainbox run``
1001- that does so unconditionally. Still, local jobs can cause harm so don't run
1002- untrusted code this way (the author of this man page recalls one local job
1003- that ran ``sudo reboot`` to measure bootchart data)
1004-
1005 Report Types
1006 ============
1007
1008@@ -74,11 +67,11 @@ plainbox-dev-analyze (1)
1009 always includes additional jobs (such as resource jobs and other
1010 dependencies)
1011
1012- The run list is of great importance. Most of the time the test operator will
1013- see tests in precisely this order. The only exception is that some test
1014- applications choose to pre-run local jobs. Still, if your job ordering is
1015- wrong in any way, inspecting the run list is the best way to debug the
1016- problem.
1017+ The run list is of great importance. Most of the time the test operator
1018+ will see tests in precisely this order. The only exception is that some
1019+ test applications choose to pre-run generator jobs (resources). Still, if
1020+ your job ordering is wrong in any way, inspecting the run list is the best
1021+ way to debug the problem.
1022
1023 See Also
1024 ========
1025diff --git a/docs/manpages/plainbox-exporter-units.rst b/docs/manpages/plainbox-exporter-units.rst
1026index 7017071..df7172c 100644
1027--- a/docs/manpages/plainbox-exporter-units.rst
1028+++ b/docs/manpages/plainbox-exporter-units.rst
1029@@ -120,36 +120,4 @@ The provider shipping such unit can be as follow::
1030    └── exporters.pxu
1031
1032 Note that exporters.pxu is not strictly needed to store the exporter units, but
1033-keeping them in a dedidated file is a good practice.
1034-
1035-How to use exporter units?
1036---------------------------
1037-
1038-In order to call an exporter unit from provider foo, you just need to add the
1039-unit id to the cli or the gui launcher in the exporter section:
1040-
1041-Example of a gui launcher:
1042-
1043- #!/usr/bin/checkbox-gui
1044-
1045- [welcome]
1046- title = "Foo"
1047- text = "bar"
1048-
1049- [exporter]
1050- HTML = "com.foo.bar::my_html"
1051-
1052-Example of a cli launcher:
1053-
1054- #!/usr/bin/env checkbox-launcher
1055- [welcome]
1056- text = Foo
1057-
1058- [suite]
1059- whitelist_filter = ^.*$
1060- whitelist_selection = ^default$
1061-
1062- [exporter]
1063- com.foo.bar::my_html
1064- com.foo.bar::my_json
1065- com.foo.baz::my_html
1066+keeping them in a dedicated file is a good practice.
1067diff --git a/docs/manpages/plainbox-file-units.rst b/docs/manpages/plainbox-file-units.rst
1068index 2a61277..3c5ddbd 100644
1069--- a/docs/manpages/plainbox-file-units.rst
1070+++ b/docs/manpages/plainbox-file-units.rst
1071@@ -36,9 +36,6 @@ There are two fields that are used by the file unit:
1072 'unit-source':
1073 The file is a source of unit definitions. Currently this is the only
1074 actually implemented value.
1075-
1076- 'legacy-whitelist':
1077- This file is a legacy whitelist.
1078
1079 'script':
1080 This file is an architecture independent executable.
1081@@ -60,4 +57,4 @@ There are two fields that are used by the file unit:
1082 This file contains copyright and licensing information.
1083
1084 'docs':
1085- This file contains documentation.
1086\ No newline at end of file
1087+ This file contains documentation.
1088diff --git a/docs/manpages/plainbox-job-units.rst b/docs/manpages/plainbox-job-units.rst
1089index be0c17f..08e8fd1 100644
1090--- a/docs/manpages/plainbox-job-units.rst
1091+++ b/docs/manpages/plainbox-job-units.rst
1092@@ -56,9 +56,6 @@ Following fields may be used by the job unit:
1093 test's outcome. This is essentially a manual job with a command.
1094 :attachment: jobs whose command output will be attached to the
1095 test report or submission.
1096- :local: a job whose command output needs to be in Checkbox job
1097- format. Jobs output by a local job will be added to the set of
1098- available jobs to be run.
1099 :resource: A job whose command output results in a set of rfc822
1100 records, containing key/value pairs, and that can be used in other
1101 jobs' ``requires`` expressions.
1102diff --git a/docs/manpages/plainbox-run.rst b/docs/manpages/plainbox-run.rst
1103index eaf5098..3363aa8 100644
1104--- a/docs/manpages/plainbox-run.rst
1105+++ b/docs/manpages/plainbox-run.rst
1106@@ -74,44 +74,6 @@ plainbox-run (1)
1107
1108 plainbox run -i '.*::foo' -i '.*::bar'
1109
1110- Selecting jobs with whitelists
1111- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1112-
1113- The second mechanism is the ``--whitelist WHITELIST`` command-line option.
1114- WhiteLists (or test plans, which is somewhat easier to relate to).
1115- Whitelists are simple text files composed of a list of regular expressions,
1116- identical to those that may be passed with the ``-i`` option.
1117-
1118- Unlike the ``-i`` option though, there are two kinds of whitelists.
1119- Standalone whitelists are not associated with any Plainbox Provider. Such
1120- whitelists can be distributed entirely separately from any other component
1121- and thus have no association with any namespace.
1122-
1123- Therefore, be fully qualified, each pattern must include both the namespace
1124- and the partial identifier components. For example, this is a valid, fully
1125- quallified whitelist::
1126-
1127- com.canonical.plainbox::stub/.*
1128-
1129- It will unambiguously select some of the jobs from the special, internal
1130- StubBox provider that is built into Plainbox. It can be saved under any
1131- filename and stored in any directory and it will always select the same set
1132- of jobs.
1133-
1134- In contrast, whitelists that are associated with a particular provider, by
1135- being stored in the per-provider ``whitelists/`` directory, carry an
1136- implicit namespace. Such whitelists are typically written without
1137- mentioning the namespace component.
1138-
1139- For example, the same "stub/.*" pattern can be abbreviated to::
1140-
1141- stub/.*
1142-
1143- Typically this syntax is used in all whitelists specific to a particular
1144- provider unless the provider maintainer explicitly wants to include a job
1145- from another namespace (for example, one of the well-known Checkbox job
1146- definitions).
1147-
1148 GENERATED JOBS
1149 ==============
1150
1151@@ -125,28 +87,7 @@ plainbox-run (1)
1152 storage devices) and then duplicate each of the store specific tests so
1153 that all devices are tested separately.
1154
1155- At this time jobs can be generated only from jobs using the plugin type
1156- `local`. Jobs of this kind are expected to print fully conforming job
1157- definitions on stdout. Generated jobs cause a few complexities and one
1158- limitation that is currently enforced is that generated jobs cannot
1159- generate additional jobs if any of the affected jobs need to run as another
1160- user.
1161-
1162- Another limitation is that jobs cannot override existing definitions.
1163-
1164- Creating Parent-Child Association
1165- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1166-
1167- A relatively niche and legacy feature of generated jobs is to print a
1168- verbatim copy of existing job definitions from a ``local`` job definition
1169- named afer a generic testing theme or category. For example the Checkbox
1170- job definition ``__wireless__`` prints, with the help of ``cat`` (1), all
1171- of the job definitions defined in the file ``wireless.txt``.
1172-
1173- This behavior is special-cased not to cause redefinition errors. Instead,
1174- existing definitions gain the ``via`` attribute that links them to the
1175- generator job. This feature is used by derivative application such as
1176- Checkbox. Plainbox is not using it at this time.
1177+ One limitation is that jobs cannot override existing definitions.
1178
1179 RESUMING
1180 ========
1181@@ -302,35 +243,24 @@ plainbox-run (1)
1182 Exported data will include comments added by the test operator to each
1183 job result that has them.
1184
1185- with-job-via:
1186- Exported data will include the ``via`` attribute alongside each job
1187- result. The via attribute contains the checksum of the job definition
1188- that generated a particular job definition. This is useful for tracking
1189- jobs generated by jobs with the plugin type `local`.
1190-
1191 with-job-hash:
1192 Exported data will include the ``hash`` attribute alongside each job
1193 result. The hash attribute is the checksum of the job definition's
1194- data. It can be useful alongside with `with-job-via`.
1195+ data.
1196
1197 machine-json:
1198 The generated JSON document will be minimal (devoid of any optional
1199 whitespace). This option is best to be used if the result is not
1200 intended to be read by humans as it saves some space.
1201
1202- rfc822
1203+ text
1204 ------
1205
1206 All of the options have the same meaning as for the `json` exporter:
1207 `with-io-log`, `squash-io-log`, `flatten-io-log`, `with-run-list`,
1208 `with-job-list`, `with-resource-map`, `with-job-defs`, `with-attachments`,
1209- `with-comments`, `with-job-via`, `with-job-hash`. The only exception is
1210- the `machine-json` option which doesn't exist for this exporter.
1211-
1212- text
1213- ----
1214-
1215- Same as with rfc822.
1216+ `with-comments`, `with-job-hash`. The only exception is the `machine-json`
1217+ option which doesn't exist for this exporter.
1218
1219 xlsx
1220 ----
1221diff --git a/docs/manpages/plainbox-test-plan-units.rst b/docs/manpages/plainbox-test-plan-units.rst
1222index 7d77e6f..6857100 100644
1223--- a/docs/manpages/plainbox-test-plan-units.rst
1224+++ b/docs/manpages/plainbox-test-plan-units.rst
1225@@ -99,11 +99,10 @@ copy such constructs when working on a new test plan from scratch
1226 common and most test plans used by Checkbox actually look like that.
1227
1228 - You can use regular expressions to select many tests at the same time.
1229- This is the only way to select generated jobs (created either by
1230- template units or by job definitions using the legacy 'local' plugin
1231- type). Please remember that the dot character has a special meaning
1232- so unless you actually want to match *any character* escape the dot
1233- with the backslash character (\\).
1234+ This is the only way to select generated jobs (created by template
1235+ units). Please remember that the dot character has a special meaning so
1236+ unless you actually want to match *any character* escape the dot with
1237+ the backslash character (\\).
1238
1239 Regardless of if you use patterns or literal job identifiers you can use
1240 their fully qualified name (the one that includes the namespace they reside
1241@@ -149,7 +148,7 @@ copy such constructs when working on a new test plan from scratch
1242
1243 Note that each entry in the bootstrap_include section must be a valid job
1244 identifier and cannot be a regular expression pattern.
1245- Also note that only local and resource jobs are allowed in this section.
1246+ Also note that only resource jobs are allowed in this section.
1247
1248 ``exclude``:
1249 A multi-line list of job identifiers or patterns matching such identifiers
1250diff --git a/docs/manpages/plainbox-trusted-launcher-1.rst b/docs/manpages/plainbox-trusted-launcher-1.rst
1251index ba889e0..77cdfeb 100644
1252--- a/docs/manpages/plainbox-trusted-launcher-1.rst
1253+++ b/docs/manpages/plainbox-trusted-launcher-1.rst
1254@@ -77,13 +77,8 @@ The following environment variables *DO NOT* affect ``plainbox-trusted-launcher-
1255 Bugs
1256 ====
1257
1258-Currently it is impossible to use ``plainbox-trusted-launcher-1`` with a
1259-``local`` job needs to run as root, that generates another ``local`` job that
1260-needs to run as root, to generate any additional jobs that also need to run as
1261-root. In other words, only one-level job generation is supported.
1262-
1263 The launcher is somewhat inefficient, in that it has to re-run all of the
1264-dependencies of the ``local`` job over and over. Ideally those would be cached,
1265+dependencies of the generator job over and over. Ideally those would be cached,
1266 per-session, but that would significantly increase the complexity of the code
1267 running as root.
1268
1269diff --git a/docs/usage.rst b/docs/usage.rst
1270index e2037b9..e2f2f0a 100644
1271--- a/docs/usage.rst
1272+++ b/docs/usage.rst
1273@@ -47,17 +47,6 @@ To list all known jobs run:
1274
1275 plainbox dev special --list-jobs
1276
1277-Running a white list
1278-^^^^^^^^^^^^^^^^^^^^
1279-
1280-To run a :term:`whitelist` pass the ``--whitelist`` or ``-w`` option.
1281-
1282-For example, to run the default white list run:
1283-
1284-.. code-block:: bash
1285-
1286- $ plainbox run -w /path/to/some/file.whitelist
1287-
1288 Saving test results
1289 ^^^^^^^^^^^^^^^^^^^
1290
1291@@ -72,7 +61,7 @@ To generate a JSON file with all of the internally available data (for storage,
1292 processing or other automation) you will need to pass three additional
1293 arguments to ``plainbox run``:
1294
1295-#. ``--output-format=com.canonical.plainbox::json``
1296+#. ``--output-format=com.canonical.com.canonical.plainbox::json``
1297 #. ``--output-options=OPTION1,OPTION2`` where *OPTIONx* are option names.
1298 #. ``--output-file=NAME`` where *NAME* is a file name.
1299
1300@@ -81,7 +70,7 @@ exporter options can be specified, separated with commas.
1301
1302 .. code-block:: bash
1303
1304- $ plainbox run --whitelist=/path/to/some/file.whitelist --output-format=com.canonical.plainbox::json --output-file=results.json
1305+ $ plainbox run -i com.canonical.certification::foo --output-format=com.canonical.plainbox::json --output-file=results.json
1306
1307 Other Exporters
1308 ---------------
1309diff --git a/plainbox/__init__.py b/plainbox/__init__.py
1310index 2a7c4e9..336f833 100644
1311--- a/plainbox/__init__.py
1312+++ b/plainbox/__init__.py
1313@@ -1,6 +1,6 @@
1314 # This file is part of Checkbox.
1315 #
1316-# Copyright 2012-2014 Canonical Ltd.
1317+# Copyright 2012-2018 Canonical Ltd.
1318 # Written by:
1319 # Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
1320 #
1321@@ -19,17 +19,11 @@
1322 :mod:`plainbox` -- main package
1323 ===============================
1324
1325-Simple checkbox redesign, without the complex message passing
1326+Simple checkbox (2008 version) redesign, without the complex message passing
1327
1328-All public API is in :mod:`plainbox.public`.
1329 All abstract base classes are in :mod:`plainbox.abc`.
1330 """
1331
1332-import sys
1333-
1334-if sys.version_info[0:2] < (3, 2):
1335- raise ImportError("plainbox requires python 3.2") # pragma: no cover
1336-
1337 # PEP440 compliant version declaration.
1338 #
1339 # This is used by @public decorator to enforce our public API guarantees.
1340diff --git a/plainbox/__main__.py b/plainbox/__main__.py
1341deleted file mode 100644
1342index 41fa1ed..0000000
1343--- a/plainbox/__main__.py
1344+++ /dev/null
1345@@ -1,32 +0,0 @@
1346-# This file is part of Checkbox.
1347-#
1348-# Copyright 2013 Canonical Ltd.
1349-# Written by:
1350-# Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
1351-#
1352-# Checkbox is free software: you can redistribute it and/or modify
1353-# it under the terms of the GNU General Public License version 3,
1354-# as published by the Free Software Foundation.
1355-#
1356-# Checkbox is distributed in the hope that it will be useful,
1357-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1358-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1359-# GNU General Public License for more details.
1360-#
1361-# You should have received a copy of the GNU General Public License
1362-# along with Checkbox. If not, see <http://www.gnu.org/licenses/>.
1363-
1364-"""
1365-:mod:`plainbox.__main__` -- execute plainbox
1366-============================================
1367-
1368-This module allows plainbox to be executed with:
1369-
1370- python3 -m plainbox
1371-"""
1372-
1373-from plainbox.public import main
1374-
1375-
1376-if __name__ == '__main__':
1377- main()
1378diff --git a/plainbox/_lazymod.py b/plainbox/_lazymod.py
1379deleted file mode 100644
1380index 0d697bb..0000000
1381--- a/plainbox/_lazymod.py
1382+++ /dev/null
1383@@ -1,138 +0,0 @@
1384-# This file is part of Checkbox.
1385-#
1386-# Copyright 2015 Canonical Ltd.
1387-# Written by:
1388-# Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
1389-#
1390-# Checkbox is free software: you can redistribute it and/or modify
1391-# it under the terms of the GNU General Public License version 3,
1392-# as published by the Free Software Foundation.
1393-#
1394-# Checkbox is distributed in the hope that it will be useful,
1395-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1396-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1397-# GNU General Public License for more details.
1398-#
1399-# You should have received a copy of the GNU General Public License
1400-# along with Checkbox. If not, see <http://www.gnu.org/licenses/>.
1401-
1402-""" Implementation of a lazy module. """
1403-
1404-import inspect
1405-import sys
1406-import types
1407-
1408-
1409-class LazyModule(types.ModuleType):
1410-
1411- """
1412- A module subclass that imports things lazily on demand.
1413-
1414- There are some special provisions to make dir() and __all__ work better so
1415- that pydoc is more informative.
1416-
1417- :ivar _lazy:
1418- A mapping of 'name' to 'callable'. The callable is called only once and
1419- defines the lazily loaded version of 'name'.
1420- :ivar _all:
1421- A set of all the "public" objects. This is exposed as the module's
1422- __all__ property. It automatically collects all the objects reported
1423- via :meth:`lazily()` and :meth:`immediate()`.
1424- :ivar _old:
1425- Reference to the old (original) module. This is kept around for python
1426- 2.x compatibility. It also seems to help with implementing __dir__()
1427- """
1428-
1429- def __init__(self, name, doc, old):
1430- """ Initialize a new lazy module. """
1431- super(LazyModule, self).__init__(name, doc)
1432- self._lazy = {}
1433- self._all = set()
1434- self._old = old
1435-
1436- def __dir__(self):
1437- """ Lazy-aware version of dir(). """
1438- if sys.version_info[0] == 3:
1439- data = super(LazyModule, self).__dir__()
1440- else:
1441- data = self.__dict__.keys()
1442- data = set(data) | self._all
1443- return sorted(data)
1444-
1445- def __getattr__(self, name):
1446- """ Lazy-aware version of getattr(). """
1447- try:
1448- callable, args = self._lazy[name]
1449- except KeyError:
1450- raise AttributeError(name)
1451- value = callable(*args)
1452- del self._lazy[name]
1453- setattr(self, name, value)
1454- return value
1455-
1456- @classmethod
1457- def shadow_normal_module(cls, mod_name=None):
1458- """
1459- Shadow a module with an instance of LazyModule.
1460-
1461- :param mod_name:
1462- Name of the module to shadow. By default this is the module that is
1463- making the call into this method. This is not hard-coded as that
1464- module might be called '__main__' if it is executed via 'python -m'
1465- :returns:
1466- A fresh instance of :class:`LazyModule`.
1467- """
1468- if mod_name is None:
1469- frame = inspect.currentframe()
1470- try:
1471- mod_name = frame.f_back.f_locals['__name__']
1472- finally:
1473- del frame
1474- orig_mod = sys.modules[mod_name]
1475- lazy_mod = cls(orig_mod.__name__, orig_mod.__doc__, orig_mod)
1476- for attr in dir(orig_mod):
1477- setattr(lazy_mod, attr, getattr(orig_mod, attr))
1478- sys.modules[mod_name] = lazy_mod
1479- return lazy_mod
1480-
1481- def lazily(self, name, callable, args):
1482- """ Load something lazily. """
1483- self._lazy[name] = callable, args
1484- self._all.add(name)
1485-
1486- def immediate(self, name, value):
1487- """ Load something immediately. """
1488- setattr(self, name, value)
1489- self._all.add(name)
1490-
1491- @property
1492- def __all__(self):
1493- """
1494- A lazy-aware version of __all__.
1495-
1496- In addition to exposing all of the original module's __all__ it also
1497- contains all the (perhaps not yet loaded) objects defined via
1498- :meth:`lazily()`
1499- """
1500- return sorted(self._all)
1501-
1502- @__all__.setter
1503- def __all__(self, value):
1504- """
1505- Setter for __all__ that just updates the internal set :ivar:`_all`.
1506-
1507- This is used by :meth:`shadow_normal_module()` which copies (assigns)
1508- all of the original module's attributes, which also assigns __all__.
1509- """
1510- self._all.update(value)
1511-
1512-
1513-def far(import_path):
1514- """ Helper for lazy imports for :meth:`LazyModule.lazily()`. """
1515- module_name, func_name = import_path.split(":", 1)
1516- module = __import__(module_name, fromlist=[''])
1517- try:
1518- return getattr(module, func_name)
1519- except AttributeError:
1520- raise NotImplementedError(
1521- "%s.%s does not exist" % (module_name, func_name))
1522diff --git a/plainbox/abc.py b/plainbox/abc.py
1523index 3315d9d..f879732 100644
1524--- a/plainbox/abc.py
1525+++ b/plainbox/abc.py
1526@@ -298,11 +298,11 @@ class IJobQualifier(metaclass=ABCMeta):
1527 can simply check if a qualifier designates a particular job (if it selects
1528 it and marks for subsequent execution). This API works fine for certain
1529 tasks but it was found that it is insufficient to implement so-called
1530- whitelist ordering, where the order of jobs in a whitelist is preserved
1531- when selecting that whitelist for execution. This spawned the second,
1532+ test plan ordering, where the order of jobs in a test plan is preserved
1533+ when selecting that test plan for execution. This spawned the second,
1534 lower-level API, that gives portable visibility into composite qualifiers
1535- (such as a whitelist) and distinct select, deselect vote so that full range
1536- of current expressiveness can be preserved.
1537+ and distinct select, deselect vote so that full range of current
1538+ expressiveness can be preserved.
1539
1540 :attr VOTE_EXCLUDE:
1541 (0) vote indicating that a job should *not* be included for
1542@@ -642,12 +642,6 @@ class IProviderBackend1(metaclass=ABCMeta):
1543 """
1544
1545 @abstractproperty
1546- def whitelists_dir(self):
1547- """
1548- Return an absolute path of the whitelist directory
1549- """
1550-
1551- @abstractproperty
1552 def data_dir(self):
1553 """
1554 absolute path of the data directory
1555@@ -758,18 +752,6 @@ class IProvider1(IProviderBackend1):
1556 """
1557
1558 @abstractproperty
1559- def whitelist_list(self):
1560- """
1561- List of loaded whitelists.
1562-
1563- .. warning::
1564- :class:`WhiteList` is currently deprecated. You should never need
1565- to access them in any new code. They are entirely replaced by
1566- :class:`TestPlan`. This property is provided for completeness and
1567- it will be **removed** once whitelists classes are no longer used.
1568- """
1569-
1570- @abstractproperty
1571 def problem_list(self):
1572 """
1573 list of problems encountered by the loading process
1574diff --git a/plainbox/impl/applogic.py b/plainbox/impl/applogic.py
1575index 6f56ed4..1d90dfd 100644
1576--- a/plainbox/impl/applogic.py
1577+++ b/plainbox/impl/applogic.py
1578@@ -48,21 +48,6 @@ def get_matching_job_list(job_list, qualifier):
1579 return select_jobs(job_list, [qualifier])
1580
1581
1582-def get_whitelist_by_name(provider_list, desired_whitelist):
1583- """
1584- Get the first whitelist matching desired_whitelist from the loaded
1585- providers
1586- """
1587- for provider in provider_list:
1588- for whitelist in provider.whitelist_list:
1589- if whitelist.name == desired_whitelist:
1590- return whitelist
1591- else:
1592- raise LookupError(
1593- _("None of the providers had a whitelist "
1594- "named '{}'").format(desired_whitelist))
1595-
1596-
1597 def run_job_if_possible(session, runner, config, job, update=True, ui=None):
1598 """
1599 Coupling point for session, runner, config and job
1600@@ -103,11 +88,6 @@ class PlainBoxConfig(config.Config):
1601 environment = config.Section(
1602 help_text=_("Environment variables for scripts and jobs"))
1603
1604- extcmd = config.Variable(
1605- section='FEATURE-FLAGS', kind=str, default="legacy",
1606- validator_list=[config.ChoiceValidator(["legacy", "glibc"])],
1607- help_text=_("Which implementation of extcmd to use"))
1608-
1609 class Meta:
1610
1611 # TODO: properly depend on xdg and use real code that also handles
1612diff --git a/plainbox/impl/buildsystems.py b/plainbox/impl/buildsystems.py
1613index 2e8d7ec..8eabeb6 100644
1614--- a/plainbox/impl/buildsystems.py
1615+++ b/plainbox/impl/buildsystems.py
1616@@ -29,13 +29,6 @@ from plainbox.abc import IBuildSystem
1617 from plainbox.impl.secure.plugins import PkgResourcesPlugInCollection
1618
1619
1620-# python3.2 doesn't have shlex.quote
1621-# so let's use the bundled copy here
1622-if not hasattr(shlex, 'quote'):
1623- from ._shlex import quote
1624- shlex.quote = quote
1625-
1626-
1627 class MakefileBuildSystem(IBuildSystem):
1628 """
1629 A build system for projects using classic makefiles
1630diff --git a/plainbox/impl/commands/__init__.py b/plainbox/impl/commands/__init__.py
1631index aa0c4de..5a26853 100644
1632--- a/plainbox/impl/commands/__init__.py
1633+++ b/plainbox/impl/commands/__init__.py
1634@@ -28,7 +28,7 @@ import logging
1635
1636 from plainbox.impl.clitools import CommandBase
1637 from plainbox.impl.clitools import ToolBase
1638-from plainbox.public import get_providers
1639+from plainbox.impl.providers import get_providers
1640
1641
1642 logger = logging.getLogger("plainbox.commands")
1643diff --git a/plainbox/impl/commands/cmd_analyze.py b/plainbox/impl/commands/cmd_analyze.py
1644index 81c1f32..d9c040e 100644
1645--- a/plainbox/impl/commands/cmd_analyze.py
1646+++ b/plainbox/impl/commands/cmd_analyze.py
1647@@ -53,17 +53,6 @@ class AnalyzeCommand(PlainBoxCommand, CheckBoxCommandMixIn):
1648 parser = subparsers.add_parser(
1649 "analyze", help=_("analyze how selected jobs would be executed"),
1650 prog="plainbox dev analyze")
1651- group = parser.add_mutually_exclusive_group()
1652- group.add_argument(
1653- '-l', '--run-local',
1654- action='store_true', dest='run_local',
1655- help=_('run all selected local jobs, required to see true data'))
1656- group.add_argument(
1657- '-L', '--skip-local',
1658- action='store_false', dest='run_local',
1659- # TRANSLATORS: please keep the word 'local' untranslated.
1660- # It designates special type of jobs, not their location.
1661- help=_('do not run local jobs'))
1662 group = parser.add_argument_group("reports")
1663 group.add_argument(
1664 '-s', '--print-stats', action='store_true',
1665diff --git a/plainbox/impl/commands/cmd_checkbox.py b/plainbox/impl/commands/cmd_checkbox.py
1666index 02fa6ca..5bc9e3c 100644
1667--- a/plainbox/impl/commands/cmd_checkbox.py
1668+++ b/plainbox/impl/commands/cmd_checkbox.py
1669@@ -62,12 +62,3 @@ class CheckBoxCommandMixIn:
1670 metavar=_("PATTERN"), default=[], dest='exclude_pattern_list',
1671 # TRANSLATORS: this is in imperative form
1672 help=_("exclude jobs matching the given regular expression"))
1673- # TODO: Find a way to handle the encoding of the file
1674- group.add_argument(
1675- '-w', '--whitelist',
1676- action="append",
1677- metavar=_("WHITELIST"),
1678- default=[],
1679- type=FileType("rt"),
1680- # TRANSLATORS: this is in imperative form
1681- help=_("load whitelist containing run patterns"))
1682diff --git a/plainbox/impl/commands/inv_analyze.py b/plainbox/impl/commands/inv_analyze.py
1683index 71d99fe..194d9ef 100644
1684--- a/plainbox/impl/commands/inv_analyze.py
1685+++ b/plainbox/impl/commands/inv_analyze.py
1686@@ -56,12 +56,6 @@ class AnalyzeInvocation(CheckBoxInvocationMixIn):
1687 self.desired_job_list)
1688
1689 def run(self):
1690- if self.ns.run_local:
1691- if self.ns.print_desired_job_list:
1692- self._print_desired_job_list()
1693- if self.ns.print_run_list:
1694- self._print_run_list()
1695- self._run_local_jobs()
1696 if self.ns.print_stats:
1697 self._print_general_stats()
1698 if self.ns.print_dependency_report:
1699@@ -89,46 +83,6 @@ class AnalyzeInvocation(CheckBoxInvocationMixIn):
1700 for job in self.session.run_list:
1701 print("{}".format(job.id))
1702
1703- def _run_local_jobs(self):
1704- print(_("[Running Local Jobs]").center(80, '='))
1705- manager = SessionManager.create_with_state(self.session)
1706- try:
1707- manager.state.metadata.title = "plainbox dev analyze session"
1708- manager.state.metadata.flags = [SessionMetaData.FLAG_INCOMPLETE]
1709- manager.checkpoint()
1710- runner = JobRunner(
1711- manager.storage.location, self.provider_list,
1712- os.path.join(manager.storage.location, 'io-logs'),
1713- command_io_delegate=self)
1714- again = True
1715- while again:
1716- for job in self.session.run_list:
1717- if job.plugin == 'local':
1718- job_state = self.session.job_state_map[job.id]
1719- if job_state.result.outcome is None:
1720- self._run_local_job(manager, runner, job, job_state)
1721- break
1722- else:
1723- again = False
1724- manager.state.metadata.flags = []
1725- manager.checkpoint()
1726- finally:
1727- manager.destroy()
1728-
1729- def _run_local_job(self, manager, runner, job, job_state):
1730- print("{job}".format(job=job.id))
1731- manager.state.metadata.running_job_name = job.id
1732- manager.checkpoint()
1733- result = runner.run_job(job, job_state, self.config)
1734- self.session.update_job_result(job, result)
1735- new_desired_job_list = self._get_matching_job_list(
1736- self.ns, self.session.job_list)
1737- new_problem_list = self.session.update_desired_job_list(
1738- new_desired_job_list)
1739- if new_problem_list:
1740- print(_("Problem list"), new_problem_list)
1741- self.problem_list.extend(new_problem_list)
1742-
1743 def _print_general_stats(self):
1744 print(_("[General Statistics]").center(80, '='))
1745 print(_("Known jobs: {}").format(len(self.session.job_list)))
1746diff --git a/plainbox/impl/commands/inv_checkbox.py b/plainbox/impl/commands/inv_checkbox.py
1747index 5136c55..1321985 100644
1748--- a/plainbox/impl/commands/inv_checkbox.py
1749+++ b/plainbox/impl/commands/inv_checkbox.py
1750@@ -33,7 +33,6 @@ from plainbox.impl.secure.origin import CommandLineTextSource
1751 from plainbox.impl.secure.origin import Origin
1752 from plainbox.impl.secure.qualifiers import RegExpJobQualifier
1753 from plainbox.impl.secure.qualifiers import select_jobs
1754-from plainbox.impl.secure.qualifiers import WhiteList
1755 from plainbox.impl.secure.rfc822 import FileTextSource
1756
1757 logger = getLogger("plainbox.commands.checkbox")
1758@@ -66,51 +65,6 @@ class CheckBoxInvocationMixIn:
1759 return list(
1760 itertools.chain(*[p.job_list for p in self.provider_list]))
1761
1762- def get_whitelist_from_file(self, filename, stream=None):
1763- """
1764- Load a whitelist from a file, with special behavior.
1765-
1766- :param filename:
1767- name of the file to load
1768- :param stream:
1769- (optional) pre-opened stream pointing at the whitelist
1770- :returns:
1771- The loaded whitelist or None if loading fails for any reason
1772-
1773- This function implements special loading behavior for whitelists that
1774- makes them inherit the implicit namespace of the provider they may be a
1775- part of. Before loading the whitelist directly from the file, all known
1776- providers are interrogated to see if any of them has a whitelist that
1777- was loaded from the same file (as indicated by os.path.realpath())
1778-
1779- The stream argument can be provided if the caller already has an open
1780- file object, which is typically the case when working with argparse.
1781- """
1782- # Look up a whitelist with the same name in any of the providers
1783- wanted_realpath = os.path.realpath(filename)
1784- for provider in self.provider_list:
1785- for whitelist in provider.whitelist_list:
1786- if (whitelist.origin is not None
1787- and whitelist.origin.source is not None
1788- and isinstance(whitelist.origin.source,
1789- FileTextSource)
1790- and os.path.realpath(
1791- whitelist.origin.source.filename) ==
1792- wanted_realpath):
1793- logger.debug(
1794- _("Using whitelist %r obtained from provider %r"),
1795- whitelist.name, provider)
1796- return whitelist
1797- # Or load it directly
1798- try:
1799- if stream is not None:
1800- return WhiteList.from_string(stream.read(), filename=filename)
1801- else:
1802- return WhiteList.from_file(filename)
1803- except Exception as exc:
1804- logger.warning(
1805- _("Unable to load whitelist %r: %s"), filename, exc)
1806-
1807 def _get_matching_job_list(self, ns, job_list):
1808 logger.debug("_get_matching_job_list(%r, %r)", ns, job_list)
1809 qualifier_list = []
1810@@ -126,12 +80,6 @@ class CheckBoxInvocationMixIn:
1811 break
1812 else:
1813 logger.debug(_("There is no test plan: %s"), ns.test_plan)
1814- # Add whitelists
1815- for whitelist_file in ns.whitelist:
1816- qualifier = self.get_whitelist_from_file(
1817- whitelist_file.name, whitelist_file)
1818- if qualifier is not None:
1819- qualifier_list.append(qualifier)
1820 # Add all the --include jobs
1821 for pattern in ns.include_pattern_list:
1822 origin = Origin(CommandLineTextSource('-i', pattern), None, None)
1823diff --git a/plainbox/impl/commands/inv_run.py b/plainbox/impl/commands/inv_run.py
1824index 1f0d5f8..c61c764 100644
1825--- a/plainbox/impl/commands/inv_run.py
1826+++ b/plainbox/impl/commands/inv_run.py
1827@@ -403,16 +403,11 @@ class RunInvocation(CheckBoxInvocationMixIn):
1828 self.metadata.flags.add(SessionMetaData.FLAG_INCOMPLETE)
1829 self.manager.checkpoint()
1830 # Select all the jobs that we are likely to run. This is the
1831- # initial selection as we haven't started any jobs yet. Local jobs
1832- # will cause that to happen again.
1833+ # initial selection as we haven't started any jobs yet.
1834 self.do_initial_job_selection()
1835 # Print out our estimates
1836 self.print_estimated_duration()
1837- # Maybe ask the secure launcher to prompt for the password now. This is
1838- # imperfect as we are going to run local jobs and we cannot see if they
1839- # might need root or not. This cannot be fixed before template jobs are
1840- # added and local jobs deprecated and removed (at least not being a
1841- # part of the session we want to execute).
1842+ # Maybe ask the secure launcher to prompt for the password now.
1843 self.maybe_warm_up_authentication()
1844 # Iterate through the run list and run jobs if possible. This function
1845 # also implements backtrack to run new jobs that were added (and
1846@@ -731,7 +726,7 @@ class RunInvocation(CheckBoxInvocationMixIn):
1847 estimated_time = 0
1848 # gather jobs that we want to run and skip the jobs that already
1849 # have result, this is only needed when we run over the list of
1850- # jobs again, after discovering new jobs via the local job output
1851+ # jobs again
1852 for job in self.state.run_list:
1853 job_state = self.state.job_state_map[job.id]
1854 if job_state.result.outcome is None:
1855@@ -758,7 +753,7 @@ class RunInvocation(CheckBoxInvocationMixIn):
1856
1857 def get_ui_for_job(self, job):
1858 if self.ns.dont_suppress_output is False and (job.plugin in (
1859- 'local', 'resource', 'attachment') or
1860+ 'resource', 'attachment') or
1861 'suppress-output' in job.get_flag_set()):
1862 return NormalUI(self.C.c, show_cmd_output=False)
1863 else:
1864diff --git a/plainbox/impl/commands/inv_special.py b/plainbox/impl/commands/inv_special.py
1865index f93d0a7..bc82fa7 100644
1866--- a/plainbox/impl/commands/inv_special.py
1867+++ b/plainbox/impl/commands/inv_special.py
1868@@ -95,9 +95,6 @@ class SpecialInvocation(CheckBoxInvocationMixIn):
1869 print('\t"{}" [shape=ellipse,color=blue];'.format(job.id))
1870 elif job.plugin == "attachment":
1871 print('\t"{}" [color=green];'.format(job.id))
1872- elif job.plugin == "local":
1873- print('\t"{}" [shape=invtriangle,color=red];'.format(
1874- job.id))
1875 elif job.plugin == "shell":
1876 print('\t"{}" [];'.format(job.id))
1877 elif job.plugin in ("manual", "user-verify", "user-interact"):
1878diff --git a/plainbox/impl/commands/test_run.py b/plainbox/impl/commands/test_run.py
1879index 19d7c9c..fe6f2a0 100644
1880--- a/plainbox/impl/commands/test_run.py
1881+++ b/plainbox/impl/commands/test_run.py
1882@@ -34,7 +34,6 @@ from inspect import cleandoc
1883 from unittest import TestCase
1884
1885 from plainbox.impl.box import main
1886-from plainbox.impl.exporter.rfc822 import RFC822SessionStateExporter
1887 from plainbox.impl.exporter.text import TextSessionStateExporter
1888 from plainbox.testing_utils.io import TestIO
1889 from plainbox.vendor.mock import patch, Mock
1890@@ -63,7 +62,7 @@ class TestRun(TestCase):
1891 usage: plainbox run [-h] [--non-interactive] [-n] [--dont-suppress-output]
1892 [-f FORMAT] [-p OPTIONS] [-o FILE] [-t TRANSPORT]
1893 [--transport-where WHERE] [--transport-options OPTIONS]
1894- [-T TEST-PLAN-ID] [-i PATTERN] [-x PATTERN] [-w WHITELIST]
1895+ [-T TEST-PLAN-ID] [-i PATTERN] [-x PATTERN]
1896
1897 optional arguments:
1898 -h, --help show this help message and exit
1899@@ -100,8 +99,6 @@ class TestRun(TestCase):
1900 include jobs matching the given regular expression
1901 -x PATTERN, --exclude-pattern PATTERN
1902 exclude jobs matching the given regular expression
1903- -w WHITELIST, --whitelist WHITELIST
1904- load whitelist containing run patterns
1905 """
1906 self.assertEqual(io.combined, cleandoc(expected) + "\n")
1907
1908@@ -133,7 +130,6 @@ class TestRun(TestCase):
1909 Available output formats:
1910 com.canonical.plainbox::html - Generate a standalone HTML
1911 com.canonical.plainbox::json - Generate JSON output
1912- com.canonical.plainbox::rfc822 - Generate RCF822 output
1913 com.canonical.plainbox::text - Generate plain text output
1914 com.canonical.plainbox::tar - Generate a tar.xz archive
1915 com.canonical.plainbox::xlsx - Generate an Excel 2007+ XLSX document
1916@@ -150,10 +146,9 @@ class TestRun(TestCase):
1917 Each format may support a different set of options
1918 com.canonical.plainbox::html:
1919 com.canonical.plainbox::json:
1920- com.canonical.plainbox::rfc822: with-io-log, squash-io-log, flatten-io-log, with-run-list, with-job-list, with-resource-map, with-job-defs, with-attachments, with-comments, with-job-via, with-job-hash, with-category-map, with-certification-status
1921- com.canonical.plainbox::text: with-io-log, squash-io-log, flatten-io-log, with-run-list, with-job-list, with-resource-map, with-job-defs, with-attachments, with-comments, with-job-via, with-job-hash, with-category-map, with-certification-status
1922+ com.canonical.plainbox::text: with-io-log, squash-io-log, flatten-io-log, with-run-list, with-job-list, with-resource-map, with-job-defs, with-attachments, with-comments, with-job-hash, with-category-map, with-certification-status
1923 com.canonical.plainbox::tar:
1924- com.canonical.plainbox::xlsx: with-sys-info, with-summary, with-job-description, with-text-attachments, with-unit-categories
1925+ com.canonical.plainbox::xlsx: with-sys-info, with-summary, with-job-description, with-text-attachments
1926 com.canonical.plainbox::global:
1927 """
1928 self.assertIn(cleandoc(expected) + "\n", io.combined)
1929diff --git a/plainbox/impl/ctrl.py b/plainbox/impl/ctrl.py
1930index 1551c0e..056ee47 100644
1931--- a/plainbox/impl/ctrl.py
1932+++ b/plainbox/impl/ctrl.py
1933@@ -21,7 +21,7 @@
1934 :mod:`plainbox.impl.ctrl` -- Controller Classes
1935 ===============================================
1936
1937-Session controller classes implement the glue between models (jobs, whitelists,
1938+Session controller classes implement the glue between models (jobs, test plans,
1939 session state) and the rest of the application. They encapsulate knowledge that
1940 used to be special-cased and sprinkled around various parts of both plainbox
1941 and particular plainbox-using applications.
1942@@ -102,9 +102,6 @@ class CheckBoxSessionStateController(ISessionStateController):
1943 true. This is expressed via the 'requires' attribute. A job will
1944 become inhibited if any of the requirement programs evaluates to
1945 value other than True.
1946- * A job may have the attribute 'plugin' equal to "local" which will
1947- cause the controller to interpret the stdout of the command as a set
1948- of job definitions.
1949 * A job may have the attribute 'plugin' equal to "resource" which will
1950 cause the controller to interpret the stdout of the command as a set
1951 of resource definitions.
1952@@ -243,8 +240,8 @@ class CheckBoxSessionStateController(ISessionStateController):
1953 Results also change the ready map (jobs that can run) because of
1954 dependency relations.
1955
1956- Some results have deeper meaning, those are results for local and
1957- resource jobs. They are discussed in detail below:
1958+ Some results have deeper meaning, those are results for resource jobs.
1959+ They are discussed in detail below:
1960
1961 Resource jobs produce resource records which are used as data to run
1962 requirement expressions against. Each time a result for a resource job
1963@@ -252,11 +249,6 @@ class CheckBoxSessionStateController(ISessionStateController):
1964 records. A new entry is created in the resource map (entirely replacing
1965 any old entries), with a list of the resources that were parsed from
1966 the IO log.
1967-
1968- Local jobs produce more jobs. Like with resource jobs, their IO log is
1969- parsed and interpreted as additional jobs. Unlike in resource jobs
1970- local jobs don't replace anything. They cannot replace an existing job
1971- with the same id.
1972 """
1973 # Store the result in job_state_map
1974 session_state.job_state_map[job.id].result = result
1975@@ -265,8 +257,6 @@ class CheckBoxSessionStateController(ISessionStateController):
1976 # Treat some jobs specially and interpret their output
1977 if job.plugin == "resource":
1978 self._process_resource_result(session_state, job, result)
1979- elif job.plugin == "local":
1980- self._process_local_result(session_state, job, result)
1981
1982 def _process_resource_result(self, session_state, job, result):
1983 """
1984@@ -333,108 +323,6 @@ class CheckBoxSessionStateController(ISessionStateController):
1985 new_unit.id]
1986 job_state.via_job = job
1987
1988- def _process_local_result(self, session_state, job, result):
1989- """
1990- Analyze a result of a CheckBox "local" job and generate
1991- additional job definitions
1992- """
1993- # First parse all records and create a list of new jobs (confusing
1994- # name, not a new list of jobs)
1995- new_job_list = []
1996- for record in gen_rfc822_records_from_io_log(job, result):
1997- # Skip non-job units as the code below is wired to work with jobs
1998- # Fixes: https://bugs.launchpad.net/plainbox/+bug/1443228
1999- if record.data.get('unit', 'job') != 'job':
2000- continue
2001- new_job = job.create_child_job_from_record(record)
2002- check_result = new_job.check()
2003- # Only ignore jobs for which check() returns an error
2004- if [c for c in check_result if c.severity == Severity.error]:
2005- logger.error(_("Ignoring invalid generated job %s"),
2006- new_job.id)
2007- else:
2008- new_job_list.append(new_job)
2009- # Then for each new job, add it to the job_list, unless it collides
2010- # with another job with the same id.
2011- for new_job in new_job_list:
2012- try:
2013- added_unit = session_state.add_unit(new_job, recompute=False)
2014- except DependencyDuplicateError as exc:
2015- # XXX: there should be a channel where such errors could be
2016- # reported back to the UI layer. Perhaps update_job_result()
2017- # could simply return a list of problems in a similar manner
2018- # how update_desired_job_list() does.
2019- logger.warning(
2020- # TRANSLATORS: keep the word "local" untranslated. It is a
2021- # special type of job that needs to be distinguished.
2022- _("Local job %s produced job %s that collides with"
2023- " an existing job %s (from %s), the new job was"
2024- " discarded"),
2025- job.id, exc.duplicate_job.id, exc.job.id, exc.job.origin)
2026- else:
2027- # Set the via_job attribute of the newly added job to point to
2028- # the generator job. This way it can be traced back to the old
2029- # __category__-style local jobs or to their corresponding
2030- # generator job in general.
2031- #
2032- # NOTE: this is the only place where we assign via_job so as
2033- # long as that holds true, we can detect and break via cycles.
2034- #
2035- # Via cycles occur whenever a job can reach itself again
2036- # through via associations. Note that the chain may be longer
2037- # than one link (A->A) and can include other jobs in the list
2038- # (A->B->C->A)
2039- #
2040- # To detect a cycle we must iterate back the via chain (and we
2041- # must do it here because we have access to job_state_map that
2042- # allows this iteration to happen) and break the cycle if we
2043- # see the job being added.
2044- job_state_map = session_state.job_state_map
2045- job_state_map[added_unit.id].via_job = job
2046- via_cycle = get_via_cycle(job_state_map, added_unit)
2047- if via_cycle:
2048- logger.warning(_("Automatically breaking via-cycle: %s"),
2049- ' -> '.join(str(cycle_job)
2050- for cycle_job in via_cycle))
2051- job_state_map[added_unit.id].via_job = None
2052-
2053-
2054-def get_via_cycle(job_state_map, job):
2055- """
2056- Find a possible cycle including via_job.
2057-
2058- :param job_state_map:
2059- A dictionary mapping job.id to a JobState object.
2060- :param via_job:
2061- Any job, start of a hypothetical via job cycle.
2062- :raises KeyError:
2063- If any of the encountered jobs are not present in job_state_map.
2064- :return:
2065- A list of jobs that represent the cycle or an empty tuple if no cycle
2066- is present. The list has the property that item[0] is item[-1]
2067-
2068- A via cycle occurs if *job* is reachable through the *via_job* by
2069- recursively following via_job connection until via_job becomes None.
2070- """
2071- cycle = []
2072- seen = set()
2073- while job is not None:
2074- cycle.append(job)
2075- seen.add(job)
2076- next_job = job_state_map[job.id].via_job
2077- if next_job in seen:
2078- break
2079- job = next_job
2080- else:
2081- return ()
2082- # Discard all the jobs leading to the cycle.
2083- # cycle = cycle[cycle.index(next_job):]
2084- # This is just to hold the promise of the return value so
2085- # that processing is easier for the caller.
2086- cycle.append(next_job)
2087- # assert cycle[0] is cycle[-1]
2088- return cycle
2089-
2090
2091 def gen_rfc822_records_from_io_log(job, result):
2092 """
2093diff --git a/plainbox/impl/depmgr.py b/plainbox/impl/depmgr.py
2094index aa7fb50..f91bc2f 100644
2095--- a/plainbox/impl/depmgr.py
2096+++ b/plainbox/impl/depmgr.py
2097@@ -31,9 +31,9 @@ Job Dependency Solver.
2098 from abc import ABCMeta
2099 from abc import abstractproperty
2100 from logging import getLogger
2101+import enum
2102
2103 from plainbox.i18n import gettext as _
2104-from plainbox.vendor import enum
2105
2106
2107 logger = getLogger("plainbox.depmgr")
2108diff --git a/plainbox/impl/exporter/__init__.py b/plainbox/impl/exporter/__init__.py
2109index 703beca..4c86c5d 100644
2110--- a/plainbox/impl/exporter/__init__.py
2111+++ b/plainbox/impl/exporter/__init__.py
2112@@ -79,7 +79,6 @@ class SessionStateExporterBase(ISessionStateExporter):
2113 OPTION_WITH_JOB_DEFS = 'with-job-defs'
2114 OPTION_WITH_ATTACHMENTS = 'with-attachments'
2115 OPTION_WITH_COMMENTS = 'with-comments'
2116- OPTION_WITH_JOB_VIA = 'with-job-via'
2117 OPTION_WITH_JOB_HASH = 'with-job-hash'
2118 OPTION_WITH_CATEGORY_MAP = 'with-category-map'
2119 OPTION_WITH_CERTIFICATION_STATUS = 'with-certification-status'
2120@@ -94,7 +93,6 @@ class SessionStateExporterBase(ISessionStateExporter):
2121 OPTION_WITH_JOB_DEFS,
2122 OPTION_WITH_ATTACHMENTS,
2123 OPTION_WITH_COMMENTS,
2124- OPTION_WITH_JOB_VIA,
2125 OPTION_WITH_JOB_HASH,
2126 OPTION_WITH_CATEGORY_MAP,
2127 OPTION_WITH_CERTIFICATION_STATUS,
2128@@ -261,13 +259,6 @@ class SessionStateExporterBase(ISessionStateExporter):
2129 data['result_map'][job_id]['comments'] = \
2130 job_state.result.comments
2131
2132- # Add Parent hash if requested
2133- if self.OPTION_WITH_JOB_VIA in self._option_list:
2134- data['result_map'][job_id]['via'] = (
2135- job_state.via_job.checksum
2136- if job_state.via_job is not None else None
2137- )
2138-
2139 # Add Job hash if requested
2140 if self.OPTION_WITH_JOB_HASH in self._option_list:
2141 data['result_map'][job_id]['hash'] = job_state.job.checksum
2142diff --git a/plainbox/impl/exporter/rfc822.py b/plainbox/impl/exporter/rfc822.py
2143deleted file mode 100644
2144index 8f3a185..0000000
2145--- a/plainbox/impl/exporter/rfc822.py
2146+++ /dev/null
2147@@ -1,48 +0,0 @@
2148-# This file is part of Checkbox.
2149-#
2150-# Copyright 2012 Canonical Ltd.
2151-# Written by:
2152-# Sylvain Pineau <sylvain.pineau@canonical.com>
2153-#
2154-# Checkbox is free software: you can redistribute it and/or modify
2155-# it under the terms of the GNU General Public License version 3,
2156-# as published by the Free Software Foundation.
2157-
2158-#
2159-# Checkbox is distributed in the hope that it will be useful,
2160-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2161-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2162-# GNU General Public License for more details.
2163-#
2164-# You should have received a copy of the GNU General Public License
2165-# along with Checkbox. If not, see <http://www.gnu.org/licenses/>.
2166-
2167-"""
2168-:mod:`plainbox.impl.exporter.rfc822` -- RFC822 exporter
2169-=======================================================
2170-
2171-.. warning::
2172-
2173- THIS MODULE DOES NOT HAVE STABLE PUBLIC API
2174-"""
2175-
2176-from collections import OrderedDict
2177-from io import StringIO
2178-
2179-from plainbox.impl.exporter import SessionStateExporterBase
2180-from plainbox.impl.secure.rfc822 import RFC822Record
2181-
2182-
2183-class RFC822SessionStateExporter(SessionStateExporterBase):
2184- """
2185- Session state exporter creating rfc822 documents
2186- """
2187-
2188- def dump(self, data, stream):
2189- entry = OrderedDict()
2190- string_stream = StringIO()
2191- for job_name, job_data in sorted(data['result_map'].items()):
2192- entry['name'] = job_name
2193- entry.update(job_data)
2194- RFC822Record(entry).dump(string_stream)
2195- stream.write(string_stream.getvalue().encode('UTF-8'))
2196diff --git a/plainbox/impl/exporter/tar.py b/plainbox/impl/exporter/tar.py
2197index 6fe8acb..6723e93 100644
2198--- a/plainbox/impl/exporter/tar.py
2199+++ b/plainbox/impl/exporter/tar.py
2200@@ -33,8 +33,8 @@ import time
2201 from plainbox.impl.exporter import SessionStateExporterBase
2202 from plainbox.impl.exporter.jinja2 import Jinja2SessionStateExporter
2203 from plainbox.impl.exporter.xlsx import XLSXSessionStateExporter
2204+from plainbox.impl.providers import get_providers
2205 from plainbox.impl.unit.exporter import ExporterUnitSupport
2206-from plainbox.public import get_providers
2207
2208
2209 class TARSessionStateExporter(SessionStateExporterBase):
2210@@ -78,7 +78,6 @@ class TARSessionStateExporter(SessionStateExporterBase):
2211 XLSXSessionStateExporter.OPTION_WITH_SUMMARY,
2212 XLSXSessionStateExporter.OPTION_WITH_DESCRIPTION,
2213 XLSXSessionStateExporter.OPTION_WITH_TEXT_ATTACHMENTS,
2214- XLSXSessionStateExporter.OPTION_WITH_UNIT_CATEGORIES
2215 ]
2216 xlsx_exporter = XLSXSessionStateExporter(options_list)
2217 xlsx_exporter.dump_from_session_manager(manager, xlsx_stream)
2218diff --git a/plainbox/impl/exporter/test_html.py b/plainbox/impl/exporter/test_html.py
2219index 9827a00..e763571 100644
2220--- a/plainbox/impl/exporter/test_html.py
2221+++ b/plainbox/impl/exporter/test_html.py
2222@@ -31,12 +31,12 @@ import io
2223 from plainbox.abc import IJobResult
2224 from plainbox.testing_utils import resource_string
2225 from plainbox.impl.exporter.jinja2 import Jinja2SessionStateExporter
2226+from plainbox.impl.providers import get_providers
2227 from plainbox.impl.resource import Resource
2228 from plainbox.impl.result import MemoryJobResult
2229 from plainbox.impl.session import SessionManager
2230 from plainbox.impl.unit.exporter import ExporterUnitSupport
2231 from plainbox.impl.unit.job import JobDefinition
2232-from plainbox.public import get_providers
2233
2234
2235 class HTMLExporterTests(TestCase):
2236diff --git a/plainbox/impl/exporter/test_init.py b/plainbox/impl/exporter/test_init.py
2237index e590ae4..8980390 100644
2238--- a/plainbox/impl/exporter/test_init.py
2239+++ b/plainbox/impl/exporter/test_init.py
2240@@ -219,7 +219,6 @@ class SessionStateExporterBaseTests(TestCase):
2241 'uncategorised')),
2242 ('outcome', 'pass'),
2243 ('comments', None),
2244- ('via', None),
2245 ('hash', '2def0c995e1b6d934c5a91286ba164'
2246 '18845da26d057bc992a2b5dfeae2e2fe91'),
2247 ('plugin', 'shell'),
2248@@ -234,7 +233,6 @@ class SessionStateExporterBaseTests(TestCase):
2249 'uncategorised')),
2250 ('outcome', 'pass'),
2251 ('comments', 'foo'),
2252- ('via', None),
2253 ('hash', 'ed19ba54624864a7c622ff7d1e8ed5'
2254 '96b1a0fddc4b78c8fb780fe41e55250e6f'),
2255 ('plugin', 'resource'),
2256diff --git a/plainbox/impl/exporter/test_rfc822.py b/plainbox/impl/exporter/test_rfc822.py
2257deleted file mode 100644
2258index b858b61..0000000
2259--- a/plainbox/impl/exporter/test_rfc822.py
2260+++ /dev/null
2261@@ -1,47 +0,0 @@
2262-# This file is part of Checkbox.
2263-#
2264-# Copyright 2012 Canonical Ltd.
2265-# Written by:
2266-# Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2267-# Daniel Manrique <roadmr@ubuntu.com>
2268-#
2269-# Checkbox is free software: you can redistribute it and/or modify
2270-# it under the terms of the GNU General Public License version 3,
2271-# as published by the Free Software Foundation.
2272-
2273-#
2274-# Checkbox is distributed in the hope that it will be useful,
2275-# but WITHOUT ANY WARRANTY; without even the implied warranty of
2276-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2277-# GNU General Public License for more details.
2278-#
2279-# You should have received a copy of the GNU General Public License
2280-# along with Checkbox. If not, see <http://www.gnu.org/licenses/>.
2281-
2282-"""
2283-plainbox.impl.exporter.test_rfc822
2284-==================================
2285-
2286-Test definitions for plainbox.impl.exporter.rfc822 module
2287-"""
2288-
2289-from io import BytesIO
2290-from unittest import TestCase
2291-
2292-from plainbox.impl.exporter.rfc822 import RFC822SessionStateExporter
2293-
2294-
2295-class RFC822SessionStateExporterTests(TestCase):
2296-
2297- def test_dump(self):
2298- exporter = RFC822SessionStateExporter()
2299- # exporter expects this data format
2300- data = {'result_map': {'job_name': {'outcome': 'fail'}}}
2301- stream = BytesIO()
2302- exporter.dump(data, stream)
2303- expected_bytes = (
2304- "name: job_name\n"
2305- "outcome: fail\n"
2306- "\n"
2307- ).encode('UTF-8')
2308- self.assertEqual(stream.getvalue(), expected_bytes)
2309diff --git a/plainbox/impl/exporter/xlsx.py b/plainbox/impl/exporter/xlsx.py
2310index 7de2e8a..e51b193 100644
2311--- a/plainbox/impl/exporter/xlsx.py
2312+++ b/plainbox/impl/exporter/xlsx.py
2313@@ -63,14 +63,12 @@ class XLSXSessionStateExporter(SessionStateExporterBase):
2314 OPTION_WITH_SUMMARY = 'with-summary'
2315 OPTION_WITH_DESCRIPTION = 'with-job-description'
2316 OPTION_WITH_TEXT_ATTACHMENTS = 'with-text-attachments'
2317- OPTION_WITH_UNIT_CATEGORIES = 'with-unit-categories'
2318
2319 SUPPORTED_OPTION_LIST = (
2320 OPTION_WITH_SYSTEM_INFO,
2321 OPTION_WITH_SUMMARY,
2322 OPTION_WITH_DESCRIPTION,
2323 OPTION_WITH_TEXT_ATTACHMENTS,
2324- OPTION_WITH_UNIT_CATEGORIES,
2325 )
2326
2327 def __init__(self, option_list=None, exporter_unit=None):
2328@@ -96,8 +94,6 @@ class XLSXSessionStateExporter(SessionStateExporterBase):
2329 SessionStateExporterBase.OPTION_FLATTEN_IO_LOG,
2330 SessionStateExporterBase.OPTION_WITH_COMMENTS,
2331 SessionStateExporterBase.OPTION_WITH_JOB_DEFS,
2332- SessionStateExporterBase.OPTION_WITH_JOB_VIA,
2333- SessionStateExporterBase.OPTION_WITH_JOB_HASH,
2334 SessionStateExporterBase.OPTION_WITH_RESOURCE_MAP,
2335 SessionStateExporterBase.OPTION_WITH_ATTACHMENTS,
2336 SessionStateExporterBase.OPTION_WITH_CATEGORY_MAP,
2337@@ -443,30 +439,6 @@ class XLSXSessionStateExporter(SessionStateExporterBase):
2338 'x_offset': 0, 'y_offset': 10, 'x_scale': 0.50, 'y_scale': 0.50
2339 })
2340
2341- def _set_category_status(self, result_map, via, child):
2342- for parent in [j for j in result_map if result_map[j]['hash'] == via]:
2343- if 'category_status' not in result_map[parent]:
2344- result_map[parent]['category_status'] = None
2345- child_status = result_map[child]['outcome']
2346- if 'category_status' in result_map[child]:
2347- child_status = result_map[child]['category_status']
2348- # Ignore categories without any child
2349- elif result_map[child]['plugin'] == 'local':
2350- continue
2351- if child_status == IJobResult.OUTCOME_FAIL:
2352- result_map[parent]['category_status'] = IJobResult.OUTCOME_FAIL
2353- elif (
2354- child_status == IJobResult.OUTCOME_PASS and
2355- result_map[parent]['category_status'] !=
2356- IJobResult.OUTCOME_FAIL
2357- ):
2358- result_map[parent]['category_status'] = IJobResult.OUTCOME_PASS
2359- elif (
2360- result_map[parent]['category_status'] not in
2361- (IJobResult.OUTCOME_PASS, IJobResult.OUTCOME_FAIL)
2362- ):
2363- result_map[parent]['category_status'] = IJobResult.OUTCOME_SKIP
2364-
2365 def _tree(self, result_map, category_map):
2366 res = {}
2367 tmp_result_map = {}
2368@@ -500,21 +472,6 @@ class XLSXSessionStateExporter(SessionStateExporterBase):
2369 result_map.update(tmp_result_map)
2370 return res, 2
2371
2372- def _legacy_tree(self, result_map, via=None, level=0, max_level=0):
2373- res = {}
2374- for job_name in [j for j in result_map if result_map[j]['via'] == via]:
2375- level += 1
2376- # Find the maximum depth of the test tree
2377- if level > max_level:
2378- max_level = level
2379- res[job_name], max_level = self._legacy_tree(
2380- result_map, result_map[job_name]['hash'], level, max_level)
2381- # Generate parent categories status
2382- if via is not None:
2383- self._set_category_status(result_map, via, job_name)
2384- level -= 1
2385- return res, max_level
2386-
2387 def _write_job(self, tree, result_map, max_level, level=0):
2388 for job, children in OrderedDict(
2389 sorted(
2390@@ -635,11 +592,8 @@ class XLSXSessionStateExporter(SessionStateExporterBase):
2391 {'hidden': True})
2392
2393 def write_results(self, data):
2394- if self.OPTION_WITH_UNIT_CATEGORIES in self._option_list:
2395- tree, max_level = self._tree(
2396- data['result_map'], data['category_map'])
2397- else:
2398- tree, max_level = self._legacy_tree(data['result_map'])
2399+ tree, max_level = self._tree(
2400+ data['result_map'], data['category_map'])
2401 self.worksheet3.write(3, 1, _('Tests Performed'), self.format03)
2402 self.worksheet3.freeze_panes(6, 0)
2403 self.worksheet3.set_tab_color('#DC4C00') # Orange
2404diff --git a/plainbox/impl/highlevel.py b/plainbox/impl/highlevel.py
2405index 4c36ae8..eeed23e 100644
2406--- a/plainbox/impl/highlevel.py
2407+++ b/plainbox/impl/highlevel.py
2408@@ -123,9 +123,6 @@ class PlainBoxObject:
2409 return self._attrs
2410
2411
2412-# NOTE: This should merge with the service object below but I didn't want
2413-# to do it right away as that would have to alter Service.__init__() and
2414-# I want to get Explorer API right first.
2415 class Explorer:
2416 """
2417 Class simplifying discovery of various PlainBox objects.
2418@@ -164,7 +161,6 @@ class Explorer:
2419 the explorer itself
2420 - all providers
2421 - all jobs
2422- - all whitelists
2423 - all executables
2424 - all repositories
2425 - all storages
2426@@ -173,7 +169,7 @@ class Explorer:
2427 self,
2428 name='service object',
2429 group="service")
2430- # Milk each provider for jobs and whitelists
2431+ # Milk each provider for jobs and test plans
2432 for provider in self.provider_list:
2433 provider_obj = PlainBoxObject(
2434 provider,
2435@@ -189,7 +185,6 @@ class Explorer:
2436 ('tr_description', provider.tr_description()),
2437 ('jobs_dir', provider.jobs_dir),
2438 ('units_dir', provider.units_dir),
2439- ('whitelists_dir', provider.whitelists_dir),
2440 ('data_dir', provider.data_dir),
2441 ('locale_dir', provider.locale_dir),
2442 ('gettext_domain', provider.gettext_domain),
2443diff --git a/plainbox/impl/launcher.py b/plainbox/impl/launcher.py
2444index c535aef..897d80c 100644
2445--- a/plainbox/impl/launcher.py
2446+++ b/plainbox/impl/launcher.py
2447@@ -65,93 +65,7 @@ class LauncherDefinition(PlainBoxConfig):
2448 :returns: LauncherDefinition instance
2449 :raises KeyError: for unknown launcher_version values
2450 """
2451- return {
2452- '1': LauncherDefinition1,
2453- config.Unset: LauncherDefinitionLegacy
2454- }[self.launcher_version]()
2455-
2456-
2457-class LauncherDefinitionLegacy(LauncherDefinition):
2458- """
2459- Launcher definition class for 'unversioned' launchers.
2460-
2461- This definition is used for launchers that do not specify
2462- 'launcher_version' in their [launcher] section.
2463- """
2464-
2465- api_flags = config.Variable(
2466- section='launcher',
2467- kind=list,
2468- default=[],
2469- help_text=_('List of feature-flags the application requires'))
2470-
2471- api_version = config.Variable(
2472- section='launcher',
2473- default='0.99',
2474- help_text=_('Version of API the launcher uses'))
2475-
2476- title = config.Variable(
2477- section="welcome",
2478- help_text=_("Application Title"))
2479-
2480- text = config.Variable(
2481- section="welcome",
2482- help_text=_("Welcome Message"))
2483-
2484- dont_suppress_output = config.Variable(
2485- section="ui", kind=bool, default=False,
2486- help_text=_("Don't suppress the output of certain job plugin types."))
2487-
2488- whitelist_filter = config.Variable(
2489- section="suite",
2490- # TODO: valid regexp text validator
2491- help_text=_("Pattern that whitelists need to match to be displayed"))
2492-
2493- whitelist_selection = config.Variable(
2494- section="suite",
2495- # TODO: valid regexp text validator
2496- help_text=_("Pattern that whitelists need to match to be selected"))
2497-
2498- skip_whitelist_selection = config.Variable(
2499- section="suite",
2500- kind=bool,
2501- default=False,
2502- help_text=_("If enabled then suite selection screen is not displayed"))
2503-
2504- skip_test_selection = config.Variable(
2505- section="suite",
2506- kind=bool,
2507- default=False,
2508- help_text=_("If enabled then test selection screen is not displayed"))
2509-
2510- input_type = config.Variable(
2511- section="submission",
2512- # TODO: probably a choice validator
2513- help_text=_("Type of the input field?"))
2514-
2515- ok_btn_text = config.Variable(
2516- section="submission",
2517- help_text=_("Label on the 'send' button"))
2518-
2519- submit_to = config.Variable(
2520- section="transport",
2521- validator_list=[config.ChoiceValidator(get_all_transports().keys())],
2522- help_text=_("Where to submit the test results to"))
2523-
2524- # TODO: Add a validator to ensure it looks like a valid URL
2525- submit_url = config.Variable(
2526- section="transport",
2527- help_text=_("HTTP endpoint to submit data to, using the"
2528- " transport specified with submit_to."))
2529-
2530- secure_id = config.Variable(
2531- section="submission",
2532- validator_list=[config.PatternValidator(SECURE_ID_PATTERN)],
2533- help_text=_("Secure ID to identify the system this"
2534- " submission belongs to."))
2535-
2536- exporter = config.Section(
2537- help_text=_("Section with only exported unit ids as keys (no values)"))
2538+ return {'1': LauncherDefinition1}[self.launcher_version]()
2539
2540
2541 class LauncherDefinition1(LauncherDefinition):
2542diff --git a/plainbox/impl/providers/__init__.py b/plainbox/impl/providers/__init__.py
2543index a6a510b..8a6b121 100644
2544--- a/plainbox/impl/providers/__init__.py
2545+++ b/plainbox/impl/providers/__init__.py
2546@@ -22,18 +22,10 @@ APIs for working with providers.
2547 :mod:`plainbox.impl.providers` -- providers package
2548 ===================================================
2549
2550-Providers are a mechanism by which PlainBox can enumerate jobs and whitelists.
2551+Providers are a mechanism by which PlainBox can enumerate jobs and test plans.
2552 Currently there are only v1 (as in version one) providers that basically have
2553 to behave as CheckBox itself (mini CheckBox forks for example)
2554
2555-There is ongoing work and discussion on V2 providers that would have a
2556-lower-level interface and would be able to define new job types, new whitelist
2557-types and generally all the next-gen semantics.
2558-
2559-PlainBox does not come with any real provider by default. PlainBox sometimes
2560-creates special dummy providers that have particular data in them for testing.
2561-
2562-
2563 V1 providers
2564 ------------
2565
2566@@ -43,9 +35,6 @@ this is also described by :class:`IProvider1`::
2567 * there is a directory with '.txt' or '.txt.in' files with RFC822-encoded
2568 job definitions. The definitions need a particular set of keys to work.
2569
2570- * there is a directory with '.whitelist' files that contain a list (one per
2571- line) of job definitions to execute.
2572-
2573 * there is a directory with additional executables (added to PATH)
2574
2575 * there is a directory with an additional python3 libraries (added to
2576diff --git a/plainbox/impl/providers/exporters/data/checkbox.html b/plainbox/impl/providers/exporters/data/checkbox.html
2577index 8b6e5c6..9bc8eba 100644
2578--- a/plainbox/impl/providers/exporters/data/checkbox.html
2579+++ b/plainbox/impl/providers/exporters/data/checkbox.html
2580@@ -225,7 +225,7 @@
2581 </tr>
2582 </thead>
2583 <tbody>
2584- {%- for job_id, job_state in job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin not in ("resource", "local", "attachment") %}
2585+ {%- for job_id, job_state in job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin not in ("resource", "attachment") %}
2586 <tr>
2587 <td>{{ job_state.job.tr_summary() }}</td>
2588 <td style='font-weight: bold; color: {{ job_state.result.outcome_meta().color_hex }}'>{{ job_state.result.outcome_meta().tr_label }}</td>
2589diff --git a/plainbox/impl/providers/exporters/data/checkbox.json b/plainbox/impl/providers/exporters/data/checkbox.json
2590index 00ed7e1..cd49b05 100644
2591--- a/plainbox/impl/providers/exporters/data/checkbox.json
2592+++ b/plainbox/impl/providers/exporters/data/checkbox.json
2593@@ -36,7 +36,7 @@
2594 },
2595 {%- endif %}
2596 "results": [
2597- {%- for job_id, job_state in job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin not in ("resource", "local", "attachment") %}
2598+ {%- for job_id, job_state in job_state_map|dictsort if job_state.result.outcome != None and job_state.job.plugin not in ("resource", "attachment") %}
2599 {
2600 "id": "{{ job_id|strip_ns }}",
2601 "name": "{{ job_state.job.tr_summary() }}",
2602diff --git a/plainbox/impl/providers/exporters/units/exporter.pxu b/plainbox/impl/providers/exporters/units/exporter.pxu
2603index 0d7ec67..65f96eb 100644
2604--- a/plainbox/impl/providers/exporters/units/exporter.pxu
2605+++ b/plainbox/impl/providers/exporters/units/exporter.pxu
2606@@ -13,12 +13,6 @@ file_extension: json
2607 data: {"template": "checkbox.json"}
2608
2609 unit: exporter
2610-id: rfc822
2611-_summary: Generate RCF822 output
2612-entry_point: rfc822
2613-file_extension: rfc822
2614-
2615-unit: exporter
2616 id: text
2617 _summary: Generate plain text output
2618 entry_point: text
2619@@ -36,7 +30,7 @@ _summary: Generate an Excel 2007+ XLSX document
2620 entry_point: xlsx
2621 file_extension: xlsx
2622 options:
2623- with-sys-info, with-summary, with-job-description, with-text-attachments, with-unit-categories
2624+ with-sys-info, with-summary, with-job-description, with-text-attachments
2625
2626 unit: exporter
2627 id: global
2628diff --git a/plainbox/impl/providers/stubbox/units/jobs/local.pxu b/plainbox/impl/providers/stubbox/units/jobs/local.pxu
2629deleted file mode 100644
2630index 0f41e6c..0000000
2631--- a/plainbox/impl/providers/stubbox/units/jobs/local.pxu
2632+++ /dev/null
2633@@ -1,9 +0,0 @@
2634-id: stub/local/true
2635-_summary: A job generated by another job
2636-# TRANSLATORS: don't translate 'local' below.
2637-_description:
2638- Check success result from shell test case (generated from a local job)
2639-plugin: shell
2640-flags: preserve-locale
2641-command: true
2642-estimated_duration: 0.1
2643diff --git a/plainbox/impl/providers/stubbox/units/jobs/multilevel.pxu b/plainbox/impl/providers/stubbox/units/jobs/multilevel.pxu
2644deleted file mode 100644
2645index f01a716..0000000
2646--- a/plainbox/impl/providers/stubbox/units/jobs/multilevel.pxu
2647+++ /dev/null
2648@@ -1,25 +0,0 @@
2649-id: stub/multilevel
2650-_summary: A generated generator job that generates two more jobs
2651-_description: Multilevel tests
2652-plugin: local
2653-flags: preserve-locale
2654-command:
2655- cat <<'EOF'
2656- id: stub/multilevel_1
2657- _summary: Generated multi-level job 1
2658- _description: This is just a sample multilevel test. Test 1.
2659- plugin: shell
2660- command: echo 1
2661- estimated_duration: 0.1
2662- EOF
2663- echo ""
2664- cat <<'EOF'
2665- id: stub/multilevel_2
2666- _summary: Generated multi-level job 2
2667- _description: This is just a sample multilevel test. Test 2.
2668- plugin: shell
2669- command: echo 2
2670- estimated_duration: 0.1
2671- EOF
2672- echo ""
2673-estimated_duration: 0.1
2674diff --git a/plainbox/impl/providers/stubbox/units/jobs/representative.pxu b/plainbox/impl/providers/stubbox/units/jobs/representative.pxu
2675index f7d7f60..8bab00d 100644
2676--- a/plainbox/impl/providers/stubbox/units/jobs/representative.pxu
2677+++ b/plainbox/impl/providers/stubbox/units/jobs/representative.pxu
2678@@ -29,15 +29,6 @@ command:
2679 estimated_duration: 0.1
2680 category_id: plugin-representative
2681
2682-id: representative/plugin/local
2683-_summary: Job with plugin=local
2684-_description: Job with plugin=local
2685-plugin: local
2686-flags: preserve-locale
2687-command: :
2688-estimated_duration: 0.1
2689-category_id: plugin-representative
2690-
2691 id: representative/plugin/attachment
2692 _summary: Job with plugin=attachment
2693 _description: Job with plugin=attachment
2694diff --git a/plainbox/impl/providers/stubbox/units/jobs/stub.pxu b/plainbox/impl/providers/stubbox/units/jobs/stub.pxu
2695index 7bdfc8d..55c1d62 100644
2696--- a/plainbox/impl/providers/stubbox/units/jobs/stub.pxu
2697+++ b/plainbox/impl/providers/stubbox/units/jobs/stub.pxu
2698@@ -328,30 +328,6 @@ command: false
2699 estimated_duration: 25
2700 category_id: split-field-representative
2701
2702-id: __local__
2703-_summary: A job generating one more job
2704-_description:
2705- This job generates the stub/local/true job
2706-plugin: local
2707-flags: preserve-locale
2708-command:
2709- shopt -s extglob
2710- cat $PLAINBOX_PROVIDER_UNITS/jobs/local.pxu
2711-estimated_duration: 0.1
2712-category_id: plugin-representative
2713-
2714-id: __multilevel__
2715-_summary: A job generating more generator jobs
2716-_description:
2717- This job generates stub/multilevel which in turn can
2718- generate stub/multilevel_1 and stub/multilevel_2
2719-plugin: local
2720-flags: preserve-locale
2721-command:
2722- shopt -s extglob
2723- cat $PLAINBOX_PROVIDER_UNITS/jobs/multilevel.pxu
2724-estimated_duration: 0.1
2725-
2726 id: stub/root
2727 _summary: A job that runs as root
2728 _description:
2729diff --git a/plainbox/impl/providers/stubbox/whitelists/stub.whitelist b/plainbox/impl/providers/stubbox/whitelists/stub.whitelist
2730deleted file mode 100644
2731index 3c06fd3..0000000
2732--- a/plainbox/impl/providers/stubbox/whitelists/stub.whitelist
2733+++ /dev/null
2734@@ -1,23 +0,0 @@
2735-# Shell job that always works
2736-stub/true
2737-# Shell job that always fails
2738-stub/false
2739-# User-* Job collection
2740-stub/user-verify
2741-stub/user-interact
2742-stub/user-interact-verify
2743-# A manual job
2744-stub/manual
2745-# A shell job with a dependency that always works
2746-stub/dependency/good
2747-# A shell job with a dependency that always fails
2748-stub/dependency/bad
2749-# A shell job that requires a resource which is available
2750-stub/requirement/good
2751-# A shell job that requires a resource which is not available
2752-stub/requirement/bad
2753-__local__
2754-stub/local/true
2755-stub/multilevel
2756-stub/multilevel.*
2757-stub/root
2758diff --git a/plainbox/impl/providers/stubbox/whitelists/stub1.whitelist b/plainbox/impl/providers/stubbox/whitelists/stub1.whitelist
2759deleted file mode 100644
2760index d33c8a4..0000000
2761--- a/plainbox/impl/providers/stubbox/whitelists/stub1.whitelist
2762+++ /dev/null
2763@@ -1,7 +0,0 @@
2764-stub/true
2765-stub/dependency/bad
2766-# stub_package
2767-stub/requirement/good
2768-stub/requirement/bad
2769-stub/multilevel
2770-stub/multilevel.*
2771diff --git a/plainbox/impl/providers/stubbox/whitelists/stub2.whitelist b/plainbox/impl/providers/stubbox/whitelists/stub2.whitelist
2772deleted file mode 100644
2773index 7785d21..0000000
2774--- a/plainbox/impl/providers/stubbox/whitelists/stub2.whitelist
2775+++ /dev/null
2776@@ -1,7 +0,0 @@
2777-stub/false
2778-stub/dependency/good
2779-stub/dependency/bad
2780-# stub_package
2781-stub/manual
2782-__local__
2783-stub/local/true
2784diff --git a/plainbox/impl/result.py b/plainbox/impl/result.py
2785index ec325e6..e31b069 100644
2786--- a/plainbox/impl/result.py
2787+++ b/plainbox/impl/result.py
2788@@ -461,30 +461,6 @@ class MemoryJobResult(_JobResultBase):
2789 " or special the IOLogRecord tuple")
2790
2791
2792-class GzipFile(gzip.GzipFile):
2793-
2794- """
2795- Subclass of GzipFile that works around missing read1() on python3.2.
2796-
2797- See: http://bugs.python.org/issue10791
2798- """
2799-
2800- def _read_gzip_header(self):
2801- """
2802- Ignore the non-compressed garbage at the end of the file
2803-
2804- See: https://bugs.python.org/issue24301
2805- """
2806-
2807- try:
2808- return super()._read_gzip_header()
2809- except OSError:
2810- return False
2811-
2812- def read1(self, n):
2813- return self.read(n)
2814-
2815-
2816 class DiskJobResult(_JobResultBase):
2817
2818 """
2819@@ -505,7 +481,7 @@ class DiskJobResult(_JobResultBase):
2820 def get_io_log(self):
2821 record_path = self.io_log_filename
2822 if record_path:
2823- with GzipFile(record_path, mode='rb') as gzip_stream, \
2824+ with gzip.GzipFile(record_path, mode='rb') as gzip_stream, \
2825 io.TextIOWrapper(gzip_stream, encoding='UTF-8') as stream:
2826 for record in IOLogRecordReader(stream):
2827 yield record
2828diff --git a/plainbox/impl/runner.py b/plainbox/impl/runner.py
2829index 8c22892..e5a6e28 100644
2830--- a/plainbox/impl/runner.py
2831+++ b/plainbox/impl/runner.py
2832@@ -275,7 +275,7 @@ class JobRunner(IJobRunner):
2833 """
2834
2835 # List of plugins that are still executed
2836- _DRY_RUN_PLUGINS = ('local', 'resource', 'attachment')
2837+ _DRY_RUN_PLUGINS = ('resource', 'attachment')
2838
2839 def __init__(self, session_dir, provider_list, jobs_io_log_dir,
2840 command_io_delegate=None, dry_run=False,
2841@@ -520,32 +520,6 @@ class JobRunner(IJobRunner):
2842 job, job_state, config).get_result()
2843 return result
2844
2845- def run_local_job(self, job, job_state, config):
2846- """
2847- Method called to run a job with plugin field equal to 'local'.
2848-
2849- The 'local' job implements the following scenario:
2850-
2851- * Maybe display the description to the user
2852- * The API states that :meth:`JobRunner.run_job()` should only be
2853- called at this time.
2854- * Run the command and wait for it to finish
2855- * Decide on the outcome based on the return code
2856- * The method ends here
2857-
2858- .. note::
2859- Local jobs are similar to resource jobs, in that the output matters
2860- more than the return code. Unlike resource jobs and attachment
2861- jobs, the output is expected to be a job definition in the
2862- canonical RFC822 format. Local jobs are discouraged (due to some
2863- complexities they introduce) but only supported way of generating
2864- additional jobs at runtime.
2865- """
2866- if job.plugin != "local":
2867- # TRANSLATORS: please keep 'plugin' untranslated
2868- raise ValueError(_("bad job plugin value"))
2869- return self._just_run_command(job, job_state, config).get_result()
2870-
2871 def run_manual_job(self, job, job_state, config):
2872 """
2873 Method called to run a job with plugin field equal to 'manual'.
2874@@ -741,7 +715,7 @@ class JobRunner(IJobRunner):
2875 delegate, io_log_gen = self._prepare_io_handling(job, config)
2876 # Create a subprocess.Popen() like object that uses the delegate
2877 # system to observe all IO as it occurs in real time.
2878- delegate_cls = self._get_delegate_cls(config)
2879+ delegate_cls = extcmd.ExternalCommandWithDelegate
2880 extcmd_popen = delegate_cls(delegate)
2881 # Stream all IOLogRecord entries to disk
2882 record_path = self.get_record_path_for_job(job)
2883@@ -892,7 +866,7 @@ class JobRunner(IJobRunner):
2884 delegate, io_log_gen = self._prepare_io_handling(job, config)
2885 # Create a subprocess.Popen() like object that uses the delegate
2886 # system to observe all IO as it occurs in real time.
2887- delegate_cls = self._get_delegate_cls(config)
2888+ delegate_cls = extcmd.ExternalCommandWithDelegate
2889 flags = 0
2890 # Use chunked IO for jobs that explicitly request this
2891 if 'use-chunked-io' in job.get_flag_set():
2892@@ -996,14 +970,3 @@ class JobRunner(IJobRunner):
2893 logger.warning(
2894 _("Please store desired files in $PLAINBOX_SESSION_SHARE and"
2895 " use regular temporary files for everything else"))
2896-
2897- def _get_delegate_cls(self, config):
2898- if (sys.version_info[0:2] >= (3, 4) and sys.platform == 'linux'
2899- and config.extcmd == "glibc"):
2900- logger.debug("Using glibc-based command runner")
2901- from plainbox.vendor.extcmd.glibc import (
2902- GlibcExternalCommandWithDelegate)
2903- return GlibcExternalCommandWithDelegate
2904- else:
2905- logger.debug("Using classic thread-based command runner")
2906- return extcmd.ExternalCommandWithDelegate
2907diff --git a/plainbox/impl/secure/launcher1.py b/plainbox/impl/secure/launcher1.py
2908index eaa1a8e..9515cc1 100644
2909--- a/plainbox/impl/secure/launcher1.py
2910+++ b/plainbox/impl/secure/launcher1.py
2911@@ -121,11 +121,7 @@ class TrustedLauncher:
2912 logging.error(
2913 _("Syntax error in record generated from %s: %s"), job, exc)
2914 else:
2915- if job.plugin == 'local':
2916- for record in record_list:
2917- job = JobDefinition.from_rfc822_record(record)
2918- job_list.append(job)
2919- elif job.plugin == 'resource':
2920+ if job.plugin == 'resource':
2921 resource_list = []
2922 for record in record_list:
2923 resource = Resource(record.data)
2924@@ -209,9 +205,9 @@ def get_parser_for_sphinx():
2925 group.add_argument(
2926 '-g', '--generator',
2927 metavar=_('CHECKSUM'),
2928- # TRANSLATORS: don't translate 'local' in the sentence below. It
2929- # denotes a special type of job, not its location.
2930- help=_('also run a job with this checksum (assuming it is a local'
2931+ # TRANSLATORS: don't translate 'resource' in the sentence below. It
2932+ # denotes a special type of job.
2933+ help=_('also run a job with this checksum (assuming it is a resource'
2934 ' job)'))
2935 group.add_argument(
2936 '-G', '--generator-environment',
2937@@ -266,7 +262,7 @@ def main(argv=None):
2938 all_providers.load()
2939 for plugin in all_providers.get_all_plugins():
2940 launcher.add_job_list(plugin.plugin_object.job_list)
2941- # Run the local job and feed the result back to the launcher
2942+ # Run the generator job and feed the result back to the launcher
2943 if ns.generator:
2944 try:
2945 generated_job_list = launcher.run_generator_job(
2946diff --git a/plainbox/impl/secure/providers/__init__.py b/plainbox/impl/secure/providers/__init__.py
2947index 8808022..066d2ee 100644
2948--- a/plainbox/impl/secure/providers/__init__.py
2949+++ b/plainbox/impl/secure/providers/__init__.py
2950@@ -21,18 +21,10 @@
2951 :mod:`plainbox.impl.secure.providers` -- providers package
2952 ==========================================================
2953
2954-Providers are a mechanism by which PlainBox can enumerate jobs and whitelists.
2955+Providers are a mechanism by which PlainBox can enumerate jobs and test plans.
2956 Currently there are only v1 (as in version one) providers that basically have
2957 to behave as CheckBox itself (mini CheckBox forks for example)
2958
2959-There is ongoing work and discussion on V2 providers that would have a
2960-lower-level interface and would be able to define new job types, new whitelist
2961-types and generally all the next-gen semantics.
2962-
2963-PlainBox does not come with any real provider by default. PlainBox sometimes
2964-creates special dummy providers that have particular data in them for testing.
2965-
2966-
2967 V1 providers
2968 ------------
2969
2970@@ -42,9 +34,6 @@ this is also described by :class:`plainbox.abc.IProvider1`::
2971 * there is a directory with '.txt' or '.txt.in' files with RFC822-encoded
2972 job definitions. The definitions need a particular set of keys to work.
2973
2974- * there is a directory with '.whitelist' files that contain a list (one per
2975- line) of job definitions to execute.
2976-
2977 * there is a directory with additional executables (added to PATH)
2978
2979 * there is a directory with an additional python3 libraries (added to
2980diff --git a/plainbox/impl/secure/providers/test_v1.py b/plainbox/impl/secure/providers/test_v1.py
2981index d54a3c2..b0eb001 100644
2982--- a/plainbox/impl/secure/providers/test_v1.py
2983+++ b/plainbox/impl/secure/providers/test_v1.py
2984@@ -37,8 +37,6 @@ from plainbox.impl.secure.providers.v1 import Provider1Definition
2985 from plainbox.impl.secure.providers.v1 import Provider1PlugIn
2986 from plainbox.impl.secure.providers.v1 import UnitPlugIn
2987 from plainbox.impl.secure.providers.v1 import VersionValidator
2988-from plainbox.impl.secure.providers.v1 import WhiteListPlugIn
2989-from plainbox.impl.secure.qualifiers import WhiteList
2990 from plainbox.impl.secure.rfc822 import FileTextSource
2991 from plainbox.impl.secure.rfc822 import Origin
2992 from plainbox.impl.unit.file import FileUnit
2993@@ -156,7 +154,6 @@ class Provider1DefinitionTests(TestCase):
2994 "gettext_domain = domain\n"
2995 "units_dir = /some/directory/units\n"
2996 "jobs_dir = /some/directory/jobs\n"
2997- "whitelists_dir = /some/directory/whitelists\n"
2998 "data_dir = /some/directory/data\n"
2999 "bin_dir = /some/directory/bin\n"
3000 "locale_dir = /some/directory/locale\n"
3001@@ -168,7 +165,6 @@ class Provider1DefinitionTests(TestCase):
3002 self.assertEqual(def_.location, Unset)
3003 self.assertEqual(def_.units_dir, "/some/directory/units")
3004 self.assertEqual(def_.jobs_dir, "/some/directory/jobs")
3005- self.assertEqual(def_.whitelists_dir, "/some/directory/whitelists")
3006 self.assertEqual(def_.data_dir, "/some/directory/data")
3007 self.assertEqual(def_.bin_dir, "/some/directory/bin")
3008 self.assertEqual(def_.locale_dir, "/some/directory/locale")
3009@@ -211,7 +207,6 @@ class Provider1DefinitionTests(TestCase):
3010 self.assertEqual(def_.location, "/some/directory")
3011 self.assertEqual(def_.units_dir, Unset)
3012 self.assertEqual(def_.jobs_dir, Unset)
3013- self.assertEqual(def_.whitelists_dir, Unset)
3014 self.assertEqual(def_.data_dir, Unset)
3015 self.assertEqual(def_.bin_dir, Unset)
3016 self.assertEqual(def_.locale_dir, Unset)
3017@@ -369,11 +364,11 @@ class Provider1DefinitionTests(TestCase):
3018
3019 def test_init_validation__foo_dir_unset(self):
3020 """
3021- verify that Provider1Definition allows 'jobs_dir', 'whitelists_dir',
3022- 'data_dir', 'bin_dir' and 'locale_dir' fields to be unset
3023+ verify that Provider1Definition allows 'jobs_dir', 'data_dir',
3024+ 'bin_dir' and 'locale_dir' fields to be unset
3025 """
3026- for attr in ('units_dir', 'jobs_dir', 'whitelists_dir', 'data_dir',
3027- 'bin_dir', 'locale_dir'):
3028+ for attr in ('units_dir', 'jobs_dir', 'data_dir', 'bin_dir',
3029+ 'locale_dir'):
3030 def_ = Provider1Definition()
3031 setattr(def_, attr, Unset)
3032 self.assertEqual(getattr(def_, attr), Unset)
3033@@ -381,11 +376,10 @@ class Provider1DefinitionTests(TestCase):
3034 def test_init_validation__foo_dir_is_empty(self):
3035 """
3036 verify that Provider1Definition ensures that 'jobs_dir',
3037- 'whitelists_dir', 'data_dir', 'bin_dir' and 'locale_dir' fields are not
3038- empty
3039+ 'data_dir', 'bin_dir' and 'locale_dir' fields are not empty
3040 """
3041- for attr in ('units_dir', 'jobs_dir', 'whitelists_dir', 'data_dir',
3042- 'bin_dir', 'locale_dir'):
3043+ for attr in ('units_dir', 'jobs_dir', 'data_dir', 'bin_dir',
3044+ 'locale_dir'):
3045 def_ = Provider1Definition()
3046 with self.assertRaises(ValidationError) as boom:
3047 setattr(def_, attr, '')
3048@@ -394,11 +388,11 @@ class Provider1DefinitionTests(TestCase):
3049 def test_init_validation__foo_dir_relative(self):
3050 """
3051 verify that Provider1Definition ensures that 'jobs_dir',
3052- 'whitelists_dir', 'data_dir', 'bin_dir' and 'locale_dir' fields are not
3053- a relative pathname
3054+ 'data_dir', 'bin_dir' and 'locale_dir' fields are not a relative
3055+ pathname
3056 """
3057- for attr in ('units_dir', 'jobs_dir', 'whitelists_dir', 'data_dir',
3058- 'bin_dir', 'locale_dir'):
3059+ for attr in ('units_dir', 'jobs_dir', 'data_dir', 'bin_dir',
3060+ 'locale_dir'):
3061 def_ = Provider1Definition()
3062 with self.assertRaises(ValidationError) as boom:
3063 setattr(def_, attr, 'some/place')
3064@@ -407,11 +401,11 @@ class Provider1DefinitionTests(TestCase):
3065 def test_init_validation__foo_dir_doesnt_exist(self):
3066 """
3067 verify that Provider1Definition ensures that 'jobs_dir',
3068- 'whitelists_dir', 'data_dir', 'bin_dir' and 'locale_dir' fields are not
3069- pointing to a non-existing directory
3070+ 'data_dir', 'bin_dir' and 'locale_dir' fields are not pointing to a
3071+ non-existing directory
3072 """
3073- for attr in ('units_dir', 'jobs_dir', 'whitelists_dir', 'data_dir',
3074- 'bin_dir', 'locale_dir'):
3075+ for attr in ('units_dir', 'jobs_dir', 'data_dir', 'bin_dir',
3076+ 'locale_dir'):
3077 def_ = Provider1Definition()
3078 with self.assertRaises(ValidationError) as boom:
3079 with mock.patch('os.path.isdir') as mock_isdir:
3080@@ -437,7 +431,6 @@ class Provider1PlugInTests(TestCase):
3081 DEF_TEXT_w_dirs = DEF_TEXT + (
3082 "units_dir = /some/directory/units\n"
3083 "jobs_dir = /some/directory/jobs\n"
3084- "whitelists_dir = /some/directory/whitelists\n"
3085 "data_dir = /some/directory/data\n"
3086 "bin_dir = /some/directory/bin\n"
3087 "locale_dir = /some/directory/locale\n"
3088@@ -488,7 +481,6 @@ class Provider1PlugInTests(TestCase):
3089 provider = self.plugin.plugin_object
3090 self.assertEqual(provider.units_dir, None)
3091 self.assertEqual(provider.jobs_dir, None)
3092- self.assertEqual(provider.whitelists_dir, None)
3093 self.assertEqual(provider.data_dir, None)
3094 self.assertEqual(provider.bin_dir, None)
3095 self.assertEqual(provider.build_bin_dir, None)
3096@@ -505,7 +497,6 @@ class Provider1PlugInTests(TestCase):
3097 provider = self.plugin_w_location.plugin_object
3098 self.assertEqual(provider.units_dir, "/some/directory/units")
3099 self.assertEqual(provider.jobs_dir, "/some/directory/jobs")
3100- self.assertEqual(provider.whitelists_dir, "/some/directory/whitelists")
3101 self.assertEqual(provider.data_dir, "/some/directory/data")
3102 self.assertEqual(provider.bin_dir, "/some/directory/bin")
3103 self.assertEqual(provider.build_bin_dir, "/some/directory/build/bin")
3104@@ -523,7 +514,6 @@ class Provider1PlugInTests(TestCase):
3105 provider = self.plugin_w_location_w_no_dirs.plugin_object
3106 self.assertEqual(provider.units_dir, None)
3107 self.assertEqual(provider.jobs_dir, None)
3108- self.assertEqual(provider.whitelists_dir, None)
3109 self.assertEqual(provider.data_dir, None)
3110 self.assertEqual(provider.bin_dir, None)
3111 self.assertEqual(provider.build_bin_dir, "/some/directory/build/bin")
3112@@ -539,7 +529,6 @@ class Provider1PlugInTests(TestCase):
3113 provider = self.plugin_w_dirs.plugin_object
3114 self.assertEqual(provider.units_dir, "/some/directory/units")
3115 self.assertEqual(provider.jobs_dir, "/some/directory/jobs")
3116- self.assertEqual(provider.whitelists_dir, "/some/directory/whitelists")
3117 self.assertEqual(provider.data_dir, "/some/directory/data")
3118 self.assertEqual(provider.bin_dir, "/some/directory/bin")
3119 self.assertEqual(provider.build_bin_dir, None)
3120@@ -548,62 +537,6 @@ class Provider1PlugInTests(TestCase):
3121 self.assertEqual(provider.base_dir, None)
3122
3123
3124-class WhiteListPlugInTests(TestCase):
3125- """
3126- Tests for WhiteListPlugIn
3127- """
3128-
3129- LOAD_TIME = 42
3130-
3131- def setUp(self):
3132- self.plugin = WhiteListPlugIn(
3133- "/path/to/some.whitelist", "foo\nbar\n", self.LOAD_TIME)
3134-
3135- def test_plugin_name(self):
3136- """
3137- verify that the WhiteListPlugIn.plugin_name property returns
3138- WhiteList.name
3139- """
3140- self.assertEqual(self.plugin.plugin_name, "some")
3141-
3142- def test_plugin_object(self):
3143- """
3144- verify that the WhiteListPlugIn.plugin_object property returns a
3145- WhiteList
3146- """
3147- self.assertIsInstance(self.plugin.plugin_object, WhiteList)
3148-
3149- def test_plugin_load_time(self):
3150- self.assertEqual(self.plugin.plugin_load_time, self.LOAD_TIME)
3151-
3152- def test_whitelist_data(self):
3153- """
3154- verify the contents of the loaded whitelist object
3155- """
3156- self.assertEqual(
3157- self.plugin.plugin_object.qualifier_list[0].pattern_text, "^foo$")
3158- self.assertEqual(
3159- self.plugin.plugin_object.qualifier_list[1].pattern_text, "^bar$")
3160- self.assertEqual(self.plugin.plugin_object.name, 'some')
3161- self.assertEqual(
3162- self.plugin.plugin_object.origin,
3163- Origin(FileTextSource('/path/to/some.whitelist'), 1, 2))
3164-
3165- def test_init_failing(self):
3166- """
3167- verify how WhiteList() initializer works if something is wrong
3168- """
3169- # The pattern is purposefully invalid
3170- with self.assertRaises(PlugInError) as boom:
3171- WhiteListPlugIn("/path/to/some.whitelist", "*", self.LOAD_TIME)
3172- # NOTE: we should have syntax error for whitelists that keeps track or
3173- # line we're at to help developers figure out where errors such as this
3174- # are coming from.
3175- self.assertEqual(
3176- str(boom.exception),
3177- ("Cannot load '/path/to/some.whitelist': nothing to repeat"))
3178-
3179-
3180 class UnitPlugInTests(TestCase):
3181 """
3182 Tests for UnitPlugIn
3183@@ -686,7 +619,6 @@ class Provider1Tests(TestCase):
3184 GETTEXT_DOMAIN = "domain"
3185 UNITS_DIR = "units-dir"
3186 JOBS_DIR = "jobs-dir"
3187- WHITELISTS_DIR = "whitelists-dir"
3188 DATA_DIR = "data-dir"
3189 BIN_DIR = "bin-dir"
3190 LOCALE_DIR = "locale-dir"
3191@@ -698,8 +630,7 @@ class Provider1Tests(TestCase):
3192 self.provider = Provider1(
3193 self.NAME, self.NAMESPACE, self.VERSION, self.DESCRIPTION,
3194 self.SECURE, self.GETTEXT_DOMAIN, self.UNITS_DIR, self.JOBS_DIR,
3195- self.WHITELISTS_DIR, self.DATA_DIR, self.BIN_DIR, self.LOCALE_DIR,
3196- self.BASE_DIR,
3197+ self.DATA_DIR, self.BIN_DIR, self.LOCALE_DIR, self.BASE_DIR,
3198 # We are using dummy job definitions so let's not shout about those
3199 # being invalid in each test
3200 validate=False)
3201@@ -762,12 +693,6 @@ class Provider1Tests(TestCase):
3202 """
3203 self.assertEqual(self.provider.jobs_dir, self.JOBS_DIR)
3204
3205- def test_whitelists_dir(self):
3206- """
3207- Verify that Provider1.whitelists_dir attribute is set correctly
3208- """
3209- self.assertEqual(self.provider.whitelists_dir, self.WHITELISTS_DIR)
3210-
3211 def test_data_dir(self):
3212 """
3213 Verify that Provider1.data_dir attribute is set correctly
3214@@ -901,8 +826,7 @@ class Provider1Tests(TestCase):
3215 Provider1(
3216 self.NAME, self.NAMESPACE, self.VERSION, self.DESCRIPTION,
3217 self.SECURE, self.GETTEXT_DOMAIN, self.UNITS_DIR, self.JOBS_DIR,
3218- self.WHITELISTS_DIR, self.DATA_DIR, self.BIN_DIR, self.LOCALE_DIR,
3219- self.BASE_DIR)
3220+ self.DATA_DIR, self.BIN_DIR, self.LOCALE_DIR, self.BASE_DIR)
3221 mock_gettext.bindtextdomain.assert_called_once_with(
3222 self.GETTEXT_DOMAIN, self.LOCALE_DIR)
3223
3224@@ -915,6 +839,6 @@ class Provider1Tests(TestCase):
3225 Provider1(
3226 self.NAME, self.NAMESPACE, self.VERSION, self.DESCRIPTION,
3227 self.SECURE, self.GETTEXT_DOMAIN, self.UNITS_DIR, self.JOBS_DIR,
3228- self.WHITELISTS_DIR, self.DATA_DIR, self.BIN_DIR, locale_dir=None,
3229+ self.DATA_DIR, self.BIN_DIR, locale_dir=None,
3230 base_dir=self.BASE_DIR)
3231 self.assertEqual(mock_gettext.bindtextdomain.call_args_list, [])
3232diff --git a/plainbox/impl/secure/providers/v1.py b/plainbox/impl/secure/providers/v1.py
3233index f1a6e58..18d3213 100644
3234--- a/plainbox/impl/secure/providers/v1.py
3235+++ b/plainbox/impl/secure/providers/v1.py
3236@@ -44,7 +44,6 @@ from plainbox.impl.secure.plugins import LazyFsPlugInCollection
3237 from plainbox.impl.secure.plugins import PlugIn
3238 from plainbox.impl.secure.plugins import PlugInError
3239 from plainbox.impl.secure.plugins import now
3240-from plainbox.impl.secure.qualifiers import WhiteList
3241 from plainbox.impl.secure.rfc822 import FileTextSource
3242 from plainbox.impl.secure.rfc822 import RFC822SyntaxError
3243 from plainbox.impl.secure.rfc822 import load_rfc822_records
3244@@ -63,17 +62,14 @@ class ProviderContentPlugIn(PlugIn):
3245 """
3246 PlugIn class for loading provider content.
3247
3248- Provider content comes in two shapes and sizes:
3249+ Provider content:
3250 - units (of any kind)
3251- - whitelists
3252
3253 The actual logic on how to load everything is encapsulated in
3254 :meth:`wrap()` though its return value is not so useful.
3255
3256 :attr unit_list:
3257 The list of loaded units
3258- :attr whitelist_list:
3259- The list of loaded whitelists
3260 """
3261
3262 def __init__(self, filename, text, load_time, provider, *,
3263@@ -94,12 +90,9 @@ class ProviderContentPlugIn(PlugIn):
3264 wrap_time = now() - start_time
3265 super().__init__(filename, inspect_result, load_time, wrap_time)
3266 self.unit_list = []
3267- self.whitelist_list = []
3268 # And load all of the content from that file
3269 self.unit_list.extend(self.discover_units(
3270 inspect_result, filename, text, provider))
3271- self.whitelist_list.extend(self.discover_whitelists(
3272- inspect_result, filename, text, provider))
3273
3274 def inspect(self, filename: str, text: str, provider: "Provider1",
3275 validate: bool, validation_kwargs: "Dict[str, Any]", check:
3276@@ -110,9 +103,8 @@ class ProviderContentPlugIn(PlugIn):
3277 :meth:`plugin_object`
3278
3279 .. note::
3280- This method must *not* access neither :attr:`unit_list` nor
3281- :attr:`whitelist_list`. If needed, it can collect its own state in
3282- private instance attributes.
3283+ This method must *not* access :attr:`unit_list`. If needed, it can
3284+ collect its own state in private instance attributes.
3285 """
3286
3287 def discover_units(
3288@@ -132,23 +124,6 @@ class ProviderContentPlugIn(PlugIn):
3289 """
3290 yield self.make_file_unit(filename, provider)
3291
3292- def discover_whitelists(
3293- self, inspect_result: "Any", filename: str, text: str,
3294- provider: "Provider1"
3295- ) -> "Iterable[WhiteList]":
3296- """
3297- Discover all whitelists that were loaded by this plug-in
3298-
3299- :param wrap_result:
3300- whatever was returned on the call to :meth:`wrap()`.
3301- :returns:
3302- an iterable of whitelists.
3303-
3304- .. note::
3305- this method is always called *after* :meth:`wrap()`.
3306- """
3307- return ()
3308-
3309 def make_file_unit(self, filename, provider, role=None, base=None):
3310 if role is None or base is None:
3311 role, base, plugin_cls = provider.classify(filename)
3312@@ -161,71 +136,6 @@ class ProviderContentPlugIn(PlugIn):
3313 virtual=True)
3314
3315
3316-class WhiteListPlugIn(ProviderContentPlugIn):
3317- """
3318- A specialized :class:`plainbox.impl.secure.plugins.IPlugIn` that loads
3319- :class:`plainbox.impl.secure.qualifiers.WhiteList` instances from a file.
3320- """
3321-
3322- def inspect(self, filename: str, text: str, provider: "Provider1",
3323- validate: bool, validation_kwargs: "Dict[str, Any]", check:
3324- bool, context: "???") -> "WhiteList":
3325- if provider is not None:
3326- implicit_namespace = provider.namespace
3327- else:
3328- implicit_namespace = None
3329- origin = Origin(FileTextSource(filename), 1, text.count('\n'))
3330- return WhiteList.from_string(
3331- text, filename=filename, origin=origin,
3332- implicit_namespace=implicit_namespace)
3333-
3334- def discover_units(
3335- self, inspect_result: "WhiteList", filename: str, text: str,
3336- provider: "Provider1"
3337- ) -> "Iterable[Unit]":
3338- if provider is not None:
3339- yield self.make_file_unit(
3340- filename, provider,
3341- # NOTE: don't guess what this file is for
3342- role=FileRole.legacy_whitelist, base=provider.whitelists_dir)
3343- yield self.make_test_plan_unit(filename, text, provider)
3344-
3345- def discover_whitelists(
3346- self, inspect_result: "WhiteList", filename: str, text: str,
3347- provider: "Provider1"
3348- ) -> "Iterable[WhiteList]":
3349- yield inspect_result
3350-
3351- def make_test_plan_unit(self, filename, text, provider):
3352- name = os.path.basename(os.path.splitext(filename)[0])
3353- origin = Origin(FileTextSource(filename), 1, text.count('\n'))
3354- field_offset_map = {'include': 0}
3355- return TestPlanUnit({
3356- 'unit': TestPlanUnit.Meta.name,
3357- 'id': name,
3358- 'name': name,
3359- 'include': str(text), # delazify content
3360- }, origin=origin, provider=provider, field_offset_map=field_offset_map,
3361- virtual=True)
3362-
3363- # NOTE: This version of __init__() exists solely so that provider can
3364- # default to None. This is still used in some places and must be supported.
3365- def __init__(self, filename, text, load_time, provider=None, *,
3366- validate=False, validation_kwargs=None,
3367- check=True, context=None):
3368- super().__init__(
3369- filename, text, load_time, provider, validate=validate,
3370- validation_kwargs=validation_kwargs, check=check, context=context)
3371-
3372- # NOTE: this version of plugin_name() is just for legacy code support
3373- @property
3374- def plugin_name(self):
3375- """
3376- plugin name, the name of the WhiteList
3377- """
3378- return self.plugin_object.name
3379-
3380-
3381 class UnitPlugIn(ProviderContentPlugIn):
3382 """
3383 A specialized :class:`plainbox.impl.secure.plugins.IPlugIn` that loads a
3384@@ -307,23 +217,6 @@ class UnitPlugIn(ProviderContentPlugIn):
3385 yield unit
3386 yield self.make_file_unit(filename, provider)
3387
3388- def discover_whitelists(
3389- self, inspect_result: "List[Unit]", filename: str, text: str,
3390- provider: "Provider1"
3391- ) -> "Iterable[WhiteList]":
3392- for unit in (unit for unit in inspect_result
3393- if unit.Meta.name == 'test plan'):
3394- if unit.include is not None:
3395- patterns = []
3396- for line in unit.include.split('\n'):
3397- if '#' in line:
3398- line = line.split('#')[0]
3399- if line:
3400- patterns.append('${}^'.format(line))
3401- yield WhiteList(
3402- patterns, name=unit.partial_id, origin=unit.origin,
3403- implicit_namespace=unit.provider.namespace)
3404-
3405 # NOTE: this version of plugin_object() is just for legacy code support
3406 @property
3407 def plugin_object(self):
3408@@ -393,8 +286,6 @@ class ProviderContentEnumerator:
3409 dir_list.append(provider.bin_dir)
3410 if provider.locale_dir:
3411 dir_list.append(provider.locale_dir)
3412- if provider.whitelists_dir:
3413- dir_list.append(provider.whitelists_dir)
3414 # Find all the files that belong to a provider
3415 self._content_collection = LazyFsPlugInCollection(
3416 dir_list, ext=None, recursive=True)
3417@@ -429,8 +320,7 @@ class ProviderContentClassifier:
3418
3419 The secondary role is to provide a hint on what PlugIn to use to load such
3420 content (as units). In practice the majority of files are loaded with the
3421- :class:`UnitPlugIn` class. Legacy ``.whitelist`` files are loaded with the
3422- :class:`WhiteListPlugIn` class instead. All other files are handled by the
3423+ :class:`UnitPlugIn` class. All other files are handled by the
3424 :class:`ProviderContentPlugIn`.
3425
3426 .. note::
3427@@ -508,8 +398,6 @@ class ProviderContentClassifier:
3428 classify_fn_list.append(self._classify_pxu_jobs)
3429 if self.provider.units_dir:
3430 classify_fn_list.append(self._classify_pxu_units)
3431- if self.provider.whitelists_dir:
3432- classify_fn_list.append(self._classify_whitelist)
3433 if self.provider.data_dir:
3434 classify_fn_list.append(self._classify_data)
3435 if self.provider.bin_dir:
3436@@ -568,13 +456,6 @@ class ProviderContentClassifier:
3437 return (FileRole.unit_source, self.provider.units_dir,
3438 UnitPlugIn)
3439
3440- def _classify_whitelist(self, filename: str):
3441- """ classify .whitelist files in whitelist_dir as whitelist """
3442- if (filename.startswith(self.provider.whitelists_dir) and
3443- filename.endswith(".whitelist")):
3444- return (FileRole.legacy_whitelist, self.provider.whitelists_dir,
3445- WhiteListPlugIn)
3446-
3447 def _classify_data(self, filename: str):
3448 """ classify files in data_dir as data """
3449 if filename.startswith(self.provider.data_dir):
3450@@ -676,7 +557,7 @@ class ProviderContentLoader:
3451 :attr is_loaded:
3452 Flag indicating if the content loader has loaded all of the content
3453 :attr unit_list:
3454- A list of loaded whitelist objects
3455+ A list of loaded units objects
3456 :attr problem_list:
3457 A list of problems experienced while loading any of the content
3458 :attr path_map:
3459@@ -691,7 +572,6 @@ class ProviderContentLoader:
3460 self.provider = provider
3461 self.is_loaded = False
3462 self.unit_list = []
3463- self.whitelist_list = []
3464 self.problem_list = []
3465 self.path_map = collections.defaultdict(list) # path -> list(unit)
3466 self.id_map = collections.defaultdict(list) # id -> list(unit)
3467@@ -720,7 +600,6 @@ class ProviderContentLoader:
3468 self.problem_list.append(exc)
3469 else:
3470 self.unit_list.extend(plugin.unit_list)
3471- self.whitelist_list.extend(plugin.whitelist_list)
3472 for unit in plugin.unit_list:
3473 if hasattr(unit.Meta.fields, 'id'):
3474 self.id_map[unit.id].append(unit)
3475@@ -732,7 +611,7 @@ class Provider1(IProvider1):
3476 """
3477 A v1 provider implementation.
3478
3479- A provider is a container of jobs and whitelists. It provides additional
3480+ A provider is a container of jobs and test plans. It provides additional
3481 meta-data and knows about location of essential directories to both load
3482 structured data and provide runtime information for job execution.
3483
3484@@ -741,8 +620,8 @@ class Provider1(IProvider1):
3485 """
3486
3487 def __init__(self, name, namespace, version, description, secure,
3488- gettext_domain, units_dir, jobs_dir, whitelists_dir, data_dir,
3489- bin_dir, locale_dir, base_dir, *, validate=False,
3490+ gettext_domain, units_dir, jobs_dir, data_dir, bin_dir,
3491+ locale_dir, base_dir, *, validate=False,
3492 validation_kwargs=None, check=True, context=None):
3493 """
3494 Initialize a provider with a set of meta-data and directories.
3495@@ -778,9 +657,6 @@ class Provider1(IProvider1):
3496 :param jobs_dir:
3497 path of the directory with job definitions
3498
3499- :param whitelists_dir:
3500- path of the directory with whitelists definitions (aka test-plans)
3501-
3502 :param data_dir:
3503 path of the directory with files used by jobs at runtime
3504
3505@@ -791,8 +667,8 @@ class Provider1(IProvider1):
3506 path of the directory with locale database (translation catalogs)
3507
3508 :param base_dir:
3509- path of the directory with (perhaps) all of jobs_dir,
3510- whitelists_dir, data_dir, bin_dir, locale_dir. This may be None.
3511+ path of the directory with (perhaps) all of jobs_dir, data_dir,
3512+ bin_dir, locale_dir. This may be None.
3513 This is also the effective value of $CHECKBOX_SHARE
3514
3515 :param validate:
3516@@ -820,7 +696,6 @@ class Provider1(IProvider1):
3517 # Directories
3518 self._units_dir = units_dir
3519 self._jobs_dir = jobs_dir
3520- self._whitelists_dir = whitelists_dir
3521 self._data_dir = data_dir
3522 self._bin_dir = bin_dir
3523 self._locale_dir = locale_dir
3524@@ -843,9 +718,6 @@ class Provider1(IProvider1):
3525 if not self._loader.is_loaded:
3526 self._loader.load(self._load_kwargs)
3527
3528- def _load_whitelists(self):
3529- self._ensure_loaded()
3530-
3531 def _load_units(self, validate, validation_kwargs, check, context):
3532 self._ensure_loaded()
3533
3534@@ -890,10 +762,10 @@ class Provider1(IProvider1):
3535 definition.description, secure,
3536 definition.effective_gettext_domain,
3537 definition.effective_units_dir, definition.effective_jobs_dir,
3538- definition.effective_whitelists_dir, definition.effective_data_dir,
3539- definition.effective_bin_dir, definition.effective_locale_dir,
3540- definition.location or None, validate=validate,
3541- validation_kwargs=validation_kwargs, check=check, context=context)
3542+ definition.effective_data_dir, definition.effective_bin_dir,
3543+ definition.effective_locale_dir, definition.location or None,
3544+ validate=validate, validation_kwargs=validation_kwargs,
3545+ check=check, context=context)
3546
3547 def __repr__(self):
3548 return "<{} name:{!r}>".format(self.__class__.__name__, self.name)
3549@@ -966,13 +838,6 @@ class Provider1(IProvider1):
3550 return self._jobs_dir
3551
3552 @property
3553- def whitelists_dir(self):
3554- """
3555- absolute path of the whitelist directory
3556- """
3557- return self._whitelists_dir
3558-
3559- @property
3560 def data_dir(self):
3561 """
3562 absolute path of the data directory
3563@@ -1002,8 +867,8 @@ class Provider1(IProvider1):
3564 @property
3565 def base_dir(self):
3566 """
3567- path of the directory with (perhaps) all of jobs_dir, whitelists_dir,
3568- data_dir, bin_dir, locale_dir. This may be None
3569+ path of the directory with (perhaps) all of jobs_dir, data_dir,
3570+ bin_dir, locale_dir. This may be None
3571 """
3572 return self._base_dir
3573
3574@@ -1136,20 +1001,6 @@ class Provider1(IProvider1):
3575 unit.role in (FileRole.script, FileRole.binary))
3576
3577 @property
3578- def whitelist_list(self):
3579- """
3580- List of loaded whitelists.
3581-
3582- .. warning::
3583- :class:`WhiteList` is currently deprecated. You should never need
3584- to access them in any new code. They are entirely replaced by
3585- :class:`TestPlan`. This property is provided for completeness and
3586- it will be **removed** once whitelists classes are no longer used.
3587- """
3588- self._ensure_loaded()
3589- return self._loader.whitelist_list
3590-
3591- @property
3592 def problem_list(self):
3593 """
3594 list of problems encountered by the loading process
3595@@ -1506,43 +1357,6 @@ class Provider1Definition(Config):
3596 if implicit is not None and os.path.isdir(implicit):
3597 return implicit
3598
3599- whitelists_dir = Variable(
3600- section='PlainBox Provider',
3601- help_text=_("Pathname of the directory with whitelists definitions"),
3602- validator_list=[
3603- # NOTE: it *can* be unset
3604- NotEmptyValidator(),
3605- AbsolutePathValidator(),
3606- ExistingDirectoryValidator(),
3607- ])
3608-
3609- @property
3610- def implicit_whitelists_dir(self):
3611- """
3612- implicit value of whitelists_dir (if Unset)
3613-
3614- The implicit value is only defined if location is not Unset. It is the
3615- 'whitelists' subdirectory of the directory that location points to.
3616- """
3617- if self.location is not Unset:
3618- return os.path.join(self.location, "whitelists")
3619-
3620- @property
3621- def effective_whitelists_dir(self):
3622- """
3623- effective value of whitelists_dir
3624-
3625- The effective value is :meth:`whitelists_dir` itself, unless it is
3626- Unset. If it is Unset the effective value is the
3627- :meth:`implicit_whitelists_dir`, if that value would be valid. The
3628- effective value may be None.
3629- """
3630- if self.whitelists_dir is not Unset:
3631- return self.whitelists_dir
3632- implicit = self.implicit_whitelists_dir
3633- if implicit is not None and os.path.isdir(implicit):
3634- return implicit
3635-
3636 data_dir = Variable(
3637 section='PlainBox Provider',
3638 help_text=_("Pathname of the directory with provider data"),
3639@@ -1708,7 +1522,6 @@ class Provider1PlugIn(PlugIn):
3640 definition.location = os.path.dirname(filename)
3641 definition.units_dir = Unset
3642 definition.jobs_dir = Unset
3643- definition.whitelists_dir = Unset
3644 definition.data_dir = Unset
3645 definition.bin_dir = Unset
3646 definition.locale_dir = Unset
3647diff --git a/plainbox/impl/secure/qualifiers.py b/plainbox/impl/secure/qualifiers.py
3648index 773d953..d1ee659 100644
3649--- a/plainbox/impl/secure/qualifiers.py
3650+++ b/plainbox/impl/secure/qualifiers.py
3651@@ -210,28 +210,6 @@ class JobIdQualifier(SimpleQualifier):
3652 self.__class__.__name__, self._id, self._inclusive)
3653
3654
3655-class NonLocalJobQualifier(SimpleQualifier):
3656- """
3657- A JobQualifier that designates only non local jobs
3658- """
3659-
3660- def __init__(self, origin, inclusive=True):
3661- super().__init__(origin, inclusive)
3662-
3663- def get_simple_match(self, job):
3664- """
3665- Check if the given job matches this qualifier.
3666-
3667- This method should not be called directly, it is an implementation
3668- detail of SimpleQualifier class.
3669- """
3670- return job.plugin != 'local'
3671-
3672- def __repr__(self):
3673- return "{0}(inclusive={1})".format(
3674- self.__class__.__name__, self._inclusive)
3675-
3676-
3677 class IMatcher(metaclass=abc.ABCMeta):
3678 """
3679 Interface for objects that perform some kind of comparison on a value
3680@@ -441,206 +419,6 @@ class NonPrimitiveQualifierOrigin(Exception):
3681 """
3682
3683
3684-# NOTE: using CompositeQualifier seems strange but it's a tested proven
3685-# component so all we have to ensure is that we read the whitelist files
3686-# correctly.
3687-class WhiteList(CompositeQualifier):
3688- """
3689- A qualifier that understands checkbox whitelist files.
3690-
3691- A whitelist file is a plain text, line oriented file. Each line represents
3692- a regular expression pattern that can be matched against the id of a job.
3693-
3694- The file can contain simple shell-style comments that begin with the pound
3695- or hash key (#). Those are ignored. Comments can span both a fraction of a
3696- line as well as the whole line.
3697-
3698- For historical reasons each pattern has an implicit '^' and '$' prepended
3699- and appended (respectively) to the actual pattern specified in the file.
3700- """
3701-
3702- def __init__(self, pattern_list, name=None, origin=None,
3703- implicit_namespace=None):
3704- """
3705- Initialize a WhiteList object with the specified list of patterns.
3706-
3707- The patterns must be already mangled with '^' and '$'.
3708- """
3709- self._name = name
3710- self._origin = origin
3711- self._implicit_namespace = implicit_namespace
3712- if implicit_namespace is not None:
3713- # If we have an implicit namespace then transform all the patterns
3714- # without the namespace operator ('::')
3715- namespace_pattern = implicit_namespace.replace('.', '\\.')
3716-
3717- def transform_pattern(maybe_partial_id_pattern):
3718- if "::" not in maybe_partial_id_pattern:
3719- return "^{}::{}$".format(
3720- namespace_pattern, maybe_partial_id_pattern[1:-1])
3721- else:
3722- return maybe_partial_id_pattern
3723- qualifier_list = [
3724- RegExpJobQualifier(
3725- transform_pattern(pattern), origin, inclusive=True)
3726- for pattern in pattern_list]
3727- else:
3728- # Otherwise just use the patterns directly
3729- qualifier_list = [
3730- RegExpJobQualifier(pattern, origin, inclusive=True)
3731- for pattern in pattern_list]
3732- super().__init__(qualifier_list)
3733-
3734- def __repr__(self):
3735- return "<{} name:{!r}>".format(self.__class__.__name__, self.name)
3736-
3737- @property
3738- def name(self):
3739- """
3740- name of this WhiteList (might be None)
3741- """
3742- return self._name
3743-
3744- @name.setter
3745- def name(self, value):
3746- """
3747- set a new name for a WhiteList
3748- """
3749- self._name = value
3750-
3751- @property
3752- def origin(self):
3753- """
3754- origin object associated with this WhiteList (might be None)
3755- """
3756- return self._origin
3757-
3758- @property
3759- def implicit_namespace(self):
3760- """
3761- namespace used to qualify patterns without explicit namespace
3762- """
3763- return self._implicit_namespace
3764-
3765- @classmethod
3766- def from_file(cls, pathname, implicit_namespace=None):
3767- """
3768- Load and initialize the WhiteList object from the specified file.
3769-
3770- :param pathname:
3771- file to load
3772- :param implicit_namespace:
3773- (optional) implicit namespace for jobs that are using partial
3774- identifiers (all jobs)
3775- :returns:
3776- a fresh WhiteList object
3777- """
3778- pattern_list, max_lineno = cls._load_patterns(pathname)
3779- name = os.path.splitext(os.path.basename(pathname))[0]
3780- origin = Origin(FileTextSource(pathname), 1, max_lineno)
3781- return cls(pattern_list, name, origin, implicit_namespace)
3782-
3783- @classmethod
3784- def from_string(cls, text, *, filename=None, name=None, origin=None,
3785- implicit_namespace=None):
3786- """
3787- Load and initialize the WhiteList object from the specified string.
3788-
3789- :param text:
3790- full text of the whitelist
3791- :param filename:
3792- (optional, keyword-only) filename from which text was read from.
3793- This simulates a call to :meth:`from_file()` which properly
3794- computes the name and origin of the whitelist.
3795- :param name:
3796- (optional) name of the whitelist, only used if filename is not
3797- specified.
3798- :param origin:
3799- (optional) origin of the whitelist, only used if a filename is not
3800- specified. If omitted a default origin value will be constructed
3801- out of UnknownTextSource instance
3802- :param implicit_namespace:
3803- (optional) implicit namespace for jobs that are using partial
3804- identifiers (all jobs)
3805- :returns:
3806- a fresh WhiteList object
3807-
3808- The optional filename or a pair of name and origin arguments may be
3809- provided in order to have additional meta-data. This is typically
3810- needed when the :meth:`from_file()` method cannot be used as the caller
3811- already has the full text of the intended file available.
3812- """
3813- _logger.debug("Loaded whitelist from %r", filename)
3814- pattern_list, max_lineno = cls._parse_patterns(text)
3815- # generate name and origin if filename is provided
3816- if filename is not None:
3817- name = WhiteList.name_from_filename(filename)
3818- origin = Origin(FileTextSource(filename), 1, max_lineno)
3819- else:
3820- # otherwise generate origin if it's not specified
3821- if origin is None:
3822- origin = Origin(UnknownTextSource(), 1, max_lineno)
3823- return cls(pattern_list, name, origin, implicit_namespace)
3824-
3825- @classmethod
3826- def name_from_filename(cls, filename):
3827- """
3828- Compute the name of a whitelist based on the name
3829- of the file it is stored in.
3830- """
3831- return os.path.splitext(os.path.basename(filename))[0]
3832-
3833- @classmethod
3834- def _parse_patterns(cls, text):
3835- """
3836- Load whitelist patterns from the specified text
3837-
3838- :param text:
3839- string of text, including newlines, to parse
3840- :returns:
3841- (pattern_list, lineno) where lineno is the final line number
3842- (1-based) and pattern_list is a list of regular expression strings
3843- parsed from the whitelist.
3844- """
3845- from plainbox.impl.xparsers import Re
3846- from plainbox.impl.xparsers import Visitor
3847- from plainbox.impl.xparsers import WhiteList
3848-
3849- class WhiteListVisitor(Visitor):
3850-
3851- def __init__(self):
3852- self.pattern_list = []
3853- self.lineno = 0
3854-
3855- def visit_Re_node(self, node: Re):
3856- self.pattern_list.append(r"^{}$".format(node.text.strip()))
3857- self.lineno = max(node.lineno, self.lineno)
3858- return super().generic_visit(node)
3859-
3860- visit_ReFixed_node = visit_Re_node
3861- visit_RePattern_node = visit_Re_node
3862- visit_ReErr_node = visit_Re_node
3863-
3864- visitor = WhiteListVisitor()
3865- visitor.visit(WhiteList.parse(text))
3866- return visitor.pattern_list, visitor.lineno
3867-
3868- @classmethod
3869- def _load_patterns(cls, pathname):
3870- """
3871- Load whitelist patterns from the specified file
3872-
3873- :param pathname:
3874- pathname of the file to load and parse
3875- :returns:
3876- (pattern_list, lineno) where lineno is the final line number
3877- (1-based) and pattern_list is a list of regular expression strings
3878- parsed from the whitelist.
3879- """
3880- with open(pathname, "rt", encoding="UTF-8") as stream:
3881- return cls._parse_patterns(stream.read())
3882-
3883-
3884 def get_flat_primitive_qualifier_list(qualifier_list):
3885 return list(itertools.chain(*[
3886 qual.get_primitive_qualifiers()
3887@@ -659,7 +437,7 @@ def select_jobs(job_list, qualifier_list):
3888 A sub-list of JobDefinition objects, selected from job_list.
3889 """
3890 # Flatten the qualifier list, so that we can see the fine structure of
3891- # composite objects, such as whitelists.
3892+ # composite objects.
3893 flat_qualifier_list = get_flat_primitive_qualifier_list(qualifier_list)
3894 # Short-circuit if there are no jobs to select. Min is used later and this
3895 # will allow us to assume that the matrix is not empty.
3896diff --git a/plainbox/impl/secure/test_launcher1.py b/plainbox/impl/secure/test_launcher1.py
3897index c270be5..247b306 100644
3898--- a/plainbox/impl/secure/test_launcher1.py
3899+++ b/plainbox/impl/secure/test_launcher1.py
3900@@ -110,33 +110,6 @@ class TrustedLauncherTests(TestCase):
3901 # Ensure that the return value of subprocess.call() is returned
3902 self.assertEqual(retval, mock_call())
3903
3904- @mock.patch.dict('os.environ', clear=True)
3905- @mock.patch('plainbox.impl.job.JobDefinition.from_rfc822_record')
3906- @mock.patch('plainbox.impl.secure.launcher1.load_rfc822_records')
3907- @mock.patch('subprocess.check_output')
3908- def test_run_local_job(self, mock_check_output, mock_load_rfc822_records,
3909- mock_from_rfc822_record):
3910- # Create a mock job and add it to the launcher
3911- job = mock.Mock(spec=JobDefinition, name='job', plugin='local')
3912- self.launcher.add_job_list([job])
3913- # Create two mock rfc822 records
3914- record1 = mock.Mock(spec=RFC822Record, name='record')
3915- record2 = mock.Mock(spec=RFC822Record, name='record')
3916- # Ensure that load_rfc822_records() returns some mocked records
3917- mock_load_rfc822_records.return_value = [record1, record2]
3918- # Run the tested method
3919- job_list = self.launcher.run_generator_job(job.checksum, None)
3920- # Ensure that we run the job command via job.shell
3921- mock_check_output.assert_called_with(
3922- [job.shell, '-c', job.command], env={}, universal_newlines=True)
3923- # Ensure that we parse all of the output
3924- mock_load_rfc822_records.assert_called_with(
3925- mock_check_output(), source=JobOutputTextSource(job))
3926- # Ensure that we return the jobs back
3927- self.assertEqual(len(job_list), 2)
3928- self.assertEqual(job_list[0], mock_from_rfc822_record(record1))
3929- self.assertEqual(job_list[1], mock_from_rfc822_record(record2))
3930-
3931
3932 class MainTests(TestCase):
3933 """
3934@@ -189,7 +162,7 @@ class MainTests(TestCase):
3935 -g CHECKSUM, --generator CHECKSUM
3936 also run a job with this checksum (assuming \
3937 it is a
3938- local job)
3939+ resource job)
3940 -G NAME=VALUE [NAME=VALUE ...], --generator-environment NAME=VALUE \
3941 [NAME=VALUE ...]
3942 environment passed to the generator job
3943@@ -257,9 +230,9 @@ it is a
3944 verify what happens when `plainbox-trusted-launcher-1` is called with
3945 both --hash and --via that both are okay and designate existing jobs.
3946 """
3947- # Create a mock (local) job, give it a predictable checksum
3948- local_job = mock.Mock(
3949- name='local_job',
3950+ # Create a mock generator job, give it a predictable checksum
3951+ generator_job = mock.Mock(
3952+ name='generator_job',
3953 spec=JobDefinition,
3954 checksum='5678')
3955 # Create a mock (target) job, give it a predictable checksum
3956@@ -267,16 +240,16 @@ it is a
3957 name='target_job',
3958 spec=JobDefinition,
3959 checksum='1234')
3960- # Ensure this local job is enumerated by the provider
3961- self.provider.job_list = [local_job]
3962- # Ensure that the target job is generated by the local job
3963- mock_launcher.run_local_job.return_value = [target_job]
3964+ # Ensure this generator job is enumerated by the provider
3965+ self.provider.job_list = [generator_job]
3966+ # Ensure that the target job is generated by the generator job
3967+ mock_launcher.run_generator_job.return_value = [target_job]
3968 # Run the program with io intercept
3969 with TestIO(combined=True) as io:
3970 retval = main(['--target=1234', '--generator=5678'])
3971- # Ensure that the local job command was invoked
3972+ # Ensure that the generator job command was invoked
3973 mock_launcher().run_generator_job.assert_called_with(
3974- local_job.checksum, None)
3975+ generator_job.checksum, None)
3976 # Ensure that the target job command was invoked
3977 mock_launcher().run_shell_from_job.assert_called_with(
3978 target_job.checksum, None)
3979diff --git a/plainbox/impl/secure/test_qualifiers.py b/plainbox/impl/secure/test_qualifiers.py
3980index b02cf2b..e177796 100644
3981--- a/plainbox/impl/secure/test_qualifiers.py
3982+++ b/plainbox/impl/secure/test_qualifiers.py
3983@@ -39,14 +39,12 @@ from plainbox.impl.secure.qualifiers import CompositeQualifier
3984 from plainbox.impl.secure.qualifiers import FieldQualifier
3985 from plainbox.impl.secure.qualifiers import IMatcher
3986 from plainbox.impl.secure.qualifiers import JobIdQualifier
3987-from plainbox.impl.secure.qualifiers import NonLocalJobQualifier
3988 from plainbox.impl.secure.qualifiers import NonPrimitiveQualifierOrigin
3989 from plainbox.impl.secure.qualifiers import OperatorMatcher
3990 from plainbox.impl.secure.qualifiers import PatternMatcher
3991 from plainbox.impl.secure.qualifiers import RegExpJobQualifier
3992 from plainbox.impl.secure.qualifiers import select_jobs
3993 from plainbox.impl.secure.qualifiers import SimpleQualifier
3994-from plainbox.impl.secure.qualifiers import WhiteList
3995 from plainbox.impl.testing_utils import make_job
3996 from plainbox.vendor import mock
3997
3998@@ -378,56 +376,6 @@ class JobIdQualifierTests(TestCase):
3999 JobIdQualifier('*', self.origin).designates(make_job('name')))
4000
4001
4002-class NonLocalJobQualifierTests(TestCase):
4003- """
4004- Test cases for NonLocalJobQualifier class
4005- """
4006-
4007- def setUp(self):
4008- self.origin = mock.Mock(name='origin', spec_set=Origin)
4009- self.qualifier = NonLocalJobQualifier(self.origin)
4010-
4011- def test_init(self):
4012- """
4013- verify that init assigns stuff to properties correctly
4014- """
4015- self.assertEqual(self.qualifier.origin, self.origin)
4016-
4017- def test_is_primitive(self):
4018- """
4019- verify that LocalJobQualifier.is_primitive is True
4020- """
4021- self.assertTrue(self.qualifier.is_primitive)
4022-
4023- def test_repr(self):
4024- """
4025- verify that NonLocalJobQualifier.__repr__() works as expected
4026- """
4027- self.assertEqual(
4028- repr(self.qualifier), "NonLocalJobQualifier(inclusive=True)")
4029-
4030- def test_get_vote(self):
4031- """
4032- verify that NonLocalJobQualifier.get_vote() works as expected
4033- """
4034- self.assertEqual(
4035- NonLocalJobQualifier(self.origin).get_vote(
4036- JobDefinition({'name': 'foo', 'plugin': 'shell'})),
4037- IJobQualifier.VOTE_INCLUDE)
4038- self.assertEqual(
4039- NonLocalJobQualifier(self.origin, inclusive=False).get_vote(
4040- JobDefinition({'name': 'foo', 'plugin': 'shell'})),
4041- IJobQualifier.VOTE_EXCLUDE)
4042- self.assertEqual(
4043- NonLocalJobQualifier(self.origin).get_vote(
4044- JobDefinition({'name': 'bar', 'plugin': 'local'})),
4045- IJobQualifier.VOTE_IGNORE)
4046- self.assertEqual(
4047- NonLocalJobQualifier(self.origin, inclusive=False).get_vote(
4048- JobDefinition({'name': 'bar', 'plugin': 'local'})),
4049- IJobQualifier.VOTE_IGNORE)
4050-
4051-
4052 class CompositeQualifierTests(TestCase):
4053 """
4054 Test cases for CompositeQualifier class
4055@@ -537,185 +485,6 @@ class CompositeQualifierTests(TestCase):
4056 CompositeQualifier([]).origin
4057
4058
4059-class WhiteListTests(TestCase):
4060- """
4061- Test cases for WhiteList class
4062- """
4063-
4064- _name = 'whitelist.txt'
4065-
4066- _content = [
4067- "# this is a comment",
4068- "foo # this is another comment",
4069- "bar",
4070- ""
4071- ]
4072-
4073- @contextmanager
4074- def mocked_file(self, name, content):
4075- m_open = mock.MagicMock(name='open', spec=open)
4076- m_stream = mock.MagicMock(spec=TextIOWrapper)
4077- m_stream.__enter__.return_value = m_stream
4078- # The next two lines are complementary, either will suffice but the
4079- # test may need changes if the code that reads stuff changes.
4080- m_stream.__iter__.side_effect = lambda: iter(content)
4081- m_stream.read.return_value = "\n".join(content)
4082- m_open.return_value = m_stream
4083- with mock.patch('plainbox.impl.secure.qualifiers.open', m_open,
4084- create=True):
4085- yield
4086- m_open.assert_called_once_with(name, "rt", encoding="UTF-8")
4087-
4088- def test_load_patterns(self):
4089- with self.mocked_file(self._name, self._content):
4090- pattern_list, max_lineno = WhiteList._load_patterns(self._name)
4091- self.assertEqual(pattern_list, ['^foo$', '^bar$'])
4092- self.assertEqual(max_lineno, 3)
4093-
4094- def test_designates(self):
4095- """
4096- verify that WhiteList.designates() works
4097- """
4098- self.assertTrue(
4099- WhiteList.from_string("foo").designates(make_job('foo')))
4100- self.assertTrue(
4101- WhiteList.from_string("foo\nbar\n").designates(make_job('foo')))
4102- self.assertTrue(
4103- WhiteList.from_string("foo\nbar\n").designates(make_job('bar')))
4104- # Note, it's not matching either!
4105- self.assertFalse(
4106- WhiteList.from_string("foo").designates(make_job('foobar')))
4107- self.assertFalse(
4108- WhiteList.from_string("bar").designates(make_job('foobar')))
4109-
4110- def test_from_file(self):
4111- """
4112- verify that WhiteList.from_file() works
4113- """
4114- with self.mocked_file(self._name, self._content):
4115- whitelist = WhiteList.from_file(self._name)
4116- # verify that the patterns are okay
4117- self.assertEqual(
4118- repr(whitelist.qualifier_list[0]),
4119- "RegExpJobQualifier('^foo$', inclusive=True)")
4120- # verify that whitelist name got set
4121- self.assertEqual(whitelist.name, "whitelist")
4122- # verify that the origin got set
4123- self.assertEqual(
4124- whitelist.origin,
4125- Origin(FileTextSource("whitelist.txt"), 1, 3))
4126-
4127- def test_from_string(self):
4128- """
4129- verify that WhiteList.from_string() works
4130- """
4131- whitelist = WhiteList.from_string("\n".join(self._content))
4132- # verify that the patterns are okay
4133- self.assertEqual(
4134- repr(whitelist.qualifier_list[0]),
4135- "RegExpJobQualifier('^foo$', inclusive=True)")
4136- # verify that whitelist name is the empty default
4137- self.assertEqual(whitelist.name, None)
4138- # verify that the origin got set to the default constructed value
4139- self.assertEqual(whitelist.origin, Origin(UnknownTextSource(), 1, 3))
4140-
4141- def test_from_empty_string(self):
4142- """
4143- verify that WhiteList.from_string("") works
4144- """
4145- WhiteList.from_string("")
4146-
4147- def test_from_string__with_name_and_origin(self):
4148- """
4149- verify that WhiteList.from_string() works when passing name and origin
4150- """
4151- # construct a whitelist with some dummy data, the names, pathnames and
4152- # line ranges are arbitrary
4153- whitelist = WhiteList.from_string(
4154- "\n".join(self._content), name="somefile",
4155- origin=Origin(FileTextSource("somefile.txt"), 1, 3))
4156- # verify that the patterns are okay
4157- self.assertEqual(
4158- repr(whitelist.qualifier_list[0]),
4159- "RegExpJobQualifier('^foo$', inclusive=True)")
4160- # verify that whitelist name is copied
4161- self.assertEqual(whitelist.name, "somefile")
4162- # verify that the origin is copied
4163- self.assertEqual(
4164- whitelist.origin, Origin(FileTextSource("somefile.txt"), 1, 3))
4165-
4166- def test_from_string__with_filename(self):
4167- """
4168- verify that WhiteList.from_string() works when passing filename
4169- """
4170- # construct a whitelist with some dummy data, the names, pathnames and
4171- # line ranges are arbitrary
4172- whitelist = WhiteList.from_string(
4173- "\n".join(self._content), filename="somefile.txt")
4174- # verify that the patterns are okay
4175- self.assertEqual(
4176- repr(whitelist.qualifier_list[0]),
4177- "RegExpJobQualifier('^foo$', inclusive=True)")
4178- # verify that whitelist name is derived from the filename
4179- self.assertEqual(whitelist.name, "somefile")
4180- # verify that the origin is properly derived from the filename
4181- self.assertEqual(
4182- whitelist.origin, Origin(FileTextSource("somefile.txt"), 1, 3))
4183-
4184- def test_repr(self):
4185- """
4186- verify that custom repr works
4187- """
4188- whitelist = WhiteList([], name="test")
4189- self.assertEqual(repr(whitelist), "<WhiteList name:'test'>")
4190-
4191- def test_name_getter(self):
4192- """
4193- verify that WhiteList.name getter works
4194- """
4195- self.assertEqual(WhiteList([], "foo").name, "foo")
4196-
4197- def test_name_setter(self):
4198- """
4199- verify that WhiteList.name setter works
4200- """
4201- whitelist = WhiteList([], "foo")
4202- whitelist.name = "bar"
4203- self.assertEqual(whitelist.name, "bar")
4204-
4205- def test_name_from_filename(self):
4206- """
4207- verify how name_from_filename() works
4208- """
4209- self.assertEqual(
4210- WhiteList.name_from_filename("some/path/foo.whitelist"), "foo")
4211- self.assertEqual(WhiteList.name_from_filename("foo.whitelist"), "foo")
4212- self.assertEqual(WhiteList.name_from_filename("foo."), "foo")
4213- self.assertEqual(WhiteList.name_from_filename("foo"), "foo")
4214- self.assertEqual(
4215- WhiteList.name_from_filename("foo.notawhitelist"), "foo")
4216-
4217- def test_namespace_behavior(self):
4218- """
4219- verify that WhiteList() correctly respects namespace declarations
4220- and uses implict_namespace to fully qualifiy all patterns
4221- """
4222- whitelist = WhiteList.from_string(
4223- "foo\n"
4224- "example\\.org::bar\n",
4225- implicit_namespace="other.example.org")
4226- # verify that the implicit namespace was recorded
4227- self.assertEqual(
4228- whitelist.implicit_namespace, "other.example.org")
4229- # verify that the patterns are okay
4230- self.assertEqual(
4231- whitelist.qualifier_list[0].pattern_text,
4232- "^other\\.example\\.org::foo$")
4233- self.assertEqual(
4234- whitelist.qualifier_list[1].pattern_text,
4235- "^example\\.org::bar$")
4236-
4237-
4238 class FunctionTests(TestCase):
4239
4240 def setUp(self):
4241diff --git a/plainbox/impl/session/assistant.py b/plainbox/impl/session/assistant.py
4242index 1ea7ad7..db60157 100644
4243--- a/plainbox/impl/session/assistant.py
4244+++ b/plainbox/impl/session/assistant.py
4245@@ -44,6 +44,7 @@ from plainbox.impl.developer import UnexpectedMethodCall
4246 from plainbox.impl.developer import UsageExpectation
4247 from plainbox.impl.jobcache import ResourceJobCache
4248 from plainbox.impl.result import JobResultBuilder
4249+from plainbox.impl.providers import get_providers
4250 from plainbox.impl.runner import JobRunner
4251 from plainbox.impl.runner import JobRunnerUIDelegate
4252 from plainbox.impl.secure.origin import Origin
4253@@ -60,7 +61,6 @@ from plainbox.impl.session.restart import detect_restart_strategy
4254 from plainbox.impl.session.storage import SessionStorageRepository
4255 from plainbox.impl.transport import OAuthTransport
4256 from plainbox.impl.transport import TransportError
4257-from plainbox.public import get_providers
4258 from plainbox.vendor import morris
4259
4260 _logger = logging.getLogger("plainbox.session.assistant")
4261@@ -837,9 +837,8 @@ class SessionAssistant:
4262
4263 During the bootstrap phase resource jobs that are associated with job
4264 templates may generate new jobs according to the information specified
4265- in the template. In addition, local jobs can generate arbitrary
4266- (unrestricted) units. Both of those mechanism are subject to the
4267- validation system (invalid units are discarded).
4268+ in the template. This mechanism is subject to the validation system
4269+ (invalid units are discarded).
4270
4271 When this method returns (which can take a while) the session is now
4272 ready for running any jobs.
4273@@ -850,8 +849,8 @@ class SessionAssistant:
4274 """
4275 UsageExpectation.of(self).enforce()
4276 # NOTE: there is next-to-none UI here as bootstrap jobs are limited to
4277- # just resource and local jobs (including their dependencies) so there
4278- # should be very little UI required.
4279+ # just resource jobs (including their dependencies) so there should be
4280+ # very little UI required.
4281 desired_job_list = select_jobs(
4282 self._context.state.job_list,
4283 [plan.get_bootstrap_qualifier() for plan in (
4284diff --git a/plainbox/impl/session/jobs.py b/plainbox/impl/session/jobs.py
4285index 22a138b..809c72c 100644
4286--- a/plainbox/impl/session/jobs.py
4287+++ b/plainbox/impl/session/jobs.py
4288@@ -27,6 +27,7 @@ particular session. The :class:`JobState` class holds references to a
4289 that prevent the job from being runnable in a particular session.
4290 """
4291
4292+from enum import IntEnum
4293 import logging
4294
4295 from plainbox.abc import IJobResult
4296@@ -35,7 +36,6 @@ from plainbox.impl import pod
4297 from plainbox.impl.resource import ResourceExpression
4298 from plainbox.impl.result import MemoryJobResult
4299 from plainbox.impl.unit.job import JobDefinition
4300-from plainbox.vendor.enum import IntEnum
4301
4302 logger = logging.getLogger("plainbox.session.jobs")
4303
4304diff --git a/plainbox/impl/session/manager.py b/plainbox/impl/session/manager.py
4305index 9a056fa..20616d0 100644
4306--- a/plainbox/impl/session/manager.py
4307+++ b/plainbox/impl/session/manager.py
4308@@ -39,6 +39,7 @@ import tempfile
4309
4310 from plainbox.i18n import gettext as _, ngettext
4311 from plainbox.impl import pod
4312+from plainbox.impl.providers import get_providers
4313 from plainbox.impl.session.resume import SessionResumeHelper
4314 from plainbox.impl.session.state import SessionDeviceContext
4315 from plainbox.impl.session.state import SessionState
4316@@ -47,7 +48,6 @@ from plainbox.impl.session.storage import SessionStorage
4317 from plainbox.impl.session.storage import SessionStorageRepository
4318 from plainbox.impl.session.suspend import SessionSuspendHelper
4319 from plainbox.impl.unit.testplan import TestPlanUnit
4320-from plainbox.public import get_providers
4321 from plainbox.vendor import morris
4322
4323 logger = logging.getLogger("plainbox.session.manager")
4324@@ -187,7 +187,7 @@ class SessionManager(pod.POD):
4325 return self.default_device_context.state
4326
4327 @classmethod
4328- def create(cls, repo=None, legacy_mode=False, prefix='pbox-'):
4329+ def create(cls, repo=None, prefix='pbox-'):
4330 """
4331 Create an empty session manager.
4332
4333@@ -205,24 +205,18 @@ class SessionManager(pod.POD):
4334 constructed with the default location.
4335 :ptype repo:
4336 :class:`~plainbox.impl.session.storage.SessionStorageRepository`.
4337- :param legacy_mode:
4338- Propagated to
4339- :meth:`~plainbox.impl.session.storage.SessionStorage.create()` to
4340- ensure that legacy (single session) mode is used.
4341- :ptype legacy_mode:
4342- bool
4343 :return:
4344 fresh :class:`SessionManager` instance
4345 """
4346 logger.debug("SessionManager.create()")
4347 if repo is None:
4348 repo = SessionStorageRepository()
4349- storage = SessionStorage.create(repo.location, legacy_mode, prefix)
4350+ storage = SessionStorage.create(repo.location, prefix)
4351 WellKnownDirsHelper(storage).populate()
4352 return cls([], storage)
4353
4354 @classmethod
4355- def create_with_state(cls, state, repo=None, legacy_mode=False):
4356+ def create_with_state(cls, state, repo=None):
4357 """
4358 Create a session manager by wrapping existing session state.
4359
4360@@ -237,26 +231,19 @@ class SessionManager(pod.POD):
4361 constructed with the default location.
4362 :ptype repo:
4363 :class:`~plainbox.impl.session.storage.SessionStorageRepository`.
4364- :param legacy_mode:
4365- Propagated to
4366- :meth:`~plainbox.impl.session.storage.SessionStorage.create()`
4367- to ensure that legacy (single session) mode is used.
4368- :ptype legacy_mode:
4369- bool
4370 :return:
4371 fresh :class:`SessionManager` instance
4372 """
4373 logger.debug("SessionManager.create_with_state()")
4374 if repo is None:
4375 repo = SessionStorageRepository()
4376- storage = SessionStorage.create(repo.location, legacy_mode)
4377+ storage = SessionStorage.create(repo.location)
4378 WellKnownDirsHelper(storage).populate()
4379 context = SessionDeviceContext(state)
4380 return cls([context], storage)
4381
4382 @classmethod
4383- def create_with_unit_list(cls, unit_list=None, repo=None,
4384- legacy_mode=False):
4385+ def create_with_unit_list(cls, unit_list=None, repo=None):
4386 """
4387 Create a session manager with a fresh session.
4388
4389@@ -272,12 +259,6 @@ class SessionManager(pod.POD):
4390 constructed with the default location.
4391 :ptype repo:
4392 :class:`~plainbox.impl.session.storage.SessionStorageRepository`.
4393- :param legacy_mode:
4394- Propagated to
4395- :meth:`~plainbox.impl.session.storage.SessionStorage.create()`
4396- to ensure that legacy (single session) mode is used.
4397- :ptype legacy_mode:
4398- bool
4399 :return:
4400 fresh :class:`SessionManager` instance
4401 """
4402@@ -287,7 +268,7 @@ class SessionManager(pod.POD):
4403 state = SessionState(unit_list)
4404 if repo is None:
4405 repo = SessionStorageRepository()
4406- storage = SessionStorage.create(repo.location, legacy_mode)
4407+ storage = SessionStorage.create(repo.location)
4408 context = SessionDeviceContext(state)
4409 WellKnownDirsHelper(storage).populate()
4410 return cls([context], storage)
4411@@ -480,7 +461,6 @@ class SessionManager(pod.POD):
4412 'com.canonical.plainbox::html': 'html',
4413 'com.canonical.plainbox::json': 'json',
4414 'com.canonical.plainbox::junit': 'junit',
4415- 'com.canonical.plainbox::rfc822': 'rfc822',
4416 'com.canonical.plainbox::tar': 'tar',
4417 'com.canonical.plainbox::text': 'text',
4418 'com.canonical.plainbox::xlsx': 'xlsx'
4419@@ -498,8 +478,7 @@ class SessionManager(pod.POD):
4420 Identifier of the exporter unit (which must have been loaded
4421 into the session device context of the first device). For
4422 backwards compatibility this can also be any of the legacy
4423- identifiers ``tar``, ``html``, ``json``, ``rfc822``, ``text`` or
4424- ``xlsx``.
4425+ identifiers ``tar``, ``html``, ``json``, ``text`` or ``xlsx``.
4426 :param option_list:
4427 (optional) A list of options to pass to the exporter. Each option
4428 is a string. Some strings may be of form 'key=value' but those are
4429diff --git a/plainbox/impl/session/state.py b/plainbox/impl/session/state.py
4430index 0631de1..c7c6601 100644
4431--- a/plainbox/impl/session/state.py
4432+++ b/plainbox/impl/session/state.py
4433@@ -645,10 +645,6 @@ class SessionState:
4434 Not all the jobs from this list are going to be executed (or selected
4435 for execution) by the user.
4436
4437- It may change at runtime because of local jobs. Note that in upcoming
4438- changes this will start out empty and will be changeable dynamically.
4439- It can still change due to local jobs but there is no API yes.
4440-
4441 This list cannot have any duplicates, if that is the case a
4442 :class:`DependencyDuplicateError` is raised. This has to be handled
4443 externally and is a sign that the job database is corrupted or has
4444@@ -660,8 +656,7 @@ class SessionState:
4445 This list contains all the known units, including all the know job
4446 definitions (and in the future, all test plans).
4447
4448- It may change at runtime because of local jobs and template
4449- instantiations.
4450+ It may change at runtime because of template instantiations.
4451
4452 :ivar dict job_state_map: mapping that tracks the state of each job
4453
4454@@ -971,7 +966,7 @@ class SessionState:
4455 if job.automated and estimate_automated is not None:
4456 if job.estimated_duration is not None:
4457 estimate_automated += job.estimated_duration
4458- elif job.plugin != 'local':
4459+ else:
4460 estimate_automated = None
4461 elif not job.automated and estimate_manual is not None:
4462 # We add a fixed extra amount of seconds to the run time
4463@@ -993,8 +988,8 @@ class SessionState:
4464 Results also change the ready map (jobs that can run) because of
4465 dependency relations.
4466
4467- Some results have deeper meaning, those are results for local and
4468- resource jobs. They are discussed in detail below:
4469+ Some results have deeper meaning, those are results for resource jobs.
4470+ They are discussed in detail below:
4471
4472 Resource jobs produce resource records which are used as data to run
4473 requirement expressions against. Each time a result for a resource job
4474@@ -1002,11 +997,6 @@ class SessionState:
4475 records. A new entry is created in the resource map (entirely replacing
4476 any old entries), with a list of the resources that were parsed from
4477 the IO log.
4478-
4479- Local jobs produce more jobs. Like with resource jobs, their IO log is
4480- parsed and interpreted as additional jobs. Unlike in resource jobs
4481- local jobs don't replace anything. They cannot replace an existing job
4482- with the same id.
4483 """
4484 job.controller.observe_result(self, job, result)
4485 self._recompute_job_readiness()
4486@@ -1108,8 +1098,7 @@ class SessionState:
4487 return new_job
4488 else:
4489 # If there is a clash report DependencyDuplicateError only when the
4490- # hashes are different. This prevents a common "problem" where
4491- # "__foo__" local jobs just load all jobs from the "foo" category.
4492+ # hashes are different.
4493 if new_job != existing_job:
4494 raise DependencyDuplicateError(existing_job, new_job)
4495 self._add_job_siblings_unit(new_job, recompute)
4496diff --git a/plainbox/impl/session/storage.py b/plainbox/impl/session/storage.py
4497index de9620a..c290a24 100644
4498--- a/plainbox/impl/session/storage.py
4499+++ b/plainbox/impl/session/storage.py
4500@@ -54,8 +54,6 @@ class SessionStorageRepository:
4501 :meth:SessionStorage.remove()`)
4502 """
4503
4504- _LAST_SESSION_SYMLINK = "last-session"
4505-
4506 def __init__(self, location=None):
4507 """
4508 Initialize new repository at the specified location.
4509@@ -75,35 +73,6 @@ class SessionStorageRepository:
4510 """
4511 return self._location
4512
4513- def get_last_storage(self):
4514- """
4515- Find the last session storage object created in this repository.
4516-
4517- :returns:
4518- SessionStorage object associated with the last session created in
4519- this repository using legacy mode.
4520-
4521- .. note::
4522- This will only return storage objects that were created using
4523- legacy mode. Nonlegacy storage objects will not be returned this
4524- way.
4525- """
4526- pathname = os.path.join(self.location, self._LAST_SESSION_SYMLINK)
4527- try:
4528- last_storage = os.readlink(pathname)
4529- except OSError:
4530- # The symlink can be gone or not be a real symlink
4531- # in that case just ignore it and return None
4532- return None
4533- else:
4534- # The link may be relative so let's ensure we know the full
4535- # pathname for the subsequent check (which may be performed
4536- # from another directory)
4537- last_storage = os.path.join(self._location, last_storage)
4538- # If the link points to a directory, assume it's okay
4539- if os.path.isdir(last_storage):
4540- return SessionStorage(last_storage)
4541-
4542 def get_storage_list(self):
4543 """
4544 Enumerate stored sessions in the repository.
4545@@ -251,7 +220,7 @@ class SessionStorage:
4546 return os.path.join(self._location, self._SESSION_FILE)
4547
4548 @classmethod
4549- def create(cls, base_dir, legacy_mode=False, prefix='pbox-'):
4550+ def create(cls, base_dir, prefix='pbox-'):
4551 """
4552 Create a new :class:`SessionStorage` in a random subdirectory
4553 of the specified base directory. The base directory is also
4554@@ -262,25 +231,9 @@ class SessionStorage:
4555 Typically the base directory should be obtained from
4556 :meth:`SessionStorageRepository.get_default_location()`
4557
4558- :param legacy_mode:
4559- If False (defaults to True) then the caller is expected to
4560- handle multiple sessions by itself.
4561-
4562 :param prefix:
4563 String which should prefix all session filenames. The prefix is
4564 sluggified before use.
4565-
4566- .. note::
4567- Legacy mode is where applications using PlainBox API can only
4568- handle one session. Creating another session replaces whatever was
4569- stored before. In non-legacy mode applications can enumerate
4570- sessions, create arbitrary number of sessions at the same time
4571- and remove sessions once they are no longer necessary.
4572-
4573- Legacy mode is implemented with a symbolic link called
4574- 'last-session' that keeps track of the last session created using
4575- ``legacy_mode=True``. When a new legacy-mode session is created
4576- the target of that symlink is read and recursively removed.
4577 """
4578 if not os.path.exists(base_dir):
4579 os.makedirs(base_dir)
4580@@ -298,57 +251,8 @@ class SessionStorage:
4581 os.mkdir(location)
4582 logger.debug(_("Created new storage in %r"), location)
4583 self = cls(location)
4584- if legacy_mode:
4585- self._replace_legacy_session(base_dir)
4586 return self
4587
4588- def _replace_legacy_session(self, base_dir):
4589- """
4590- Remove the previous legacy session and update the 'last-session'
4591- symlink so that it points to this session storage directory.
4592- """
4593- symlink_pathname = os.path.join(
4594- base_dir, SessionStorageRepository._LAST_SESSION_SYMLINK)
4595- # Try to read and remove the storage referenced to by last-session
4596- # symlink. This can fail if the link file is gone (which is harmless)
4597- # or when it is not an actual symlink (which means that the
4598- # repository is corrupted).
4599- try:
4600- symlink_target = os.readlink(symlink_pathname)
4601- except OSError as exc:
4602- if exc.errno == errno.ENOENT:
4603- pass
4604- elif exc.errno == errno.EINVAL:
4605- logger.warning(
4606- _("%r is not a symlink, repository %r must be corrupted"),
4607- symlink_pathname, base_dir)
4608- else:
4609- logger.warning(
4610- _("Unable to read symlink target from %r: %r"),
4611- symlink_pathname, exc)
4612- else:
4613- logger.debug(
4614- _("Removing storage associated with last session %r"),
4615- symlink_target)
4616- # Remove the old session, note that the symlink may be broken so
4617- # let's ignore any errors here
4618- shutil.rmtree(symlink_target, ignore_errors=True)
4619- # Remove the last-session symlink itself
4620- logger.debug(
4621- _("Removing symlink associated with last session: %r"),
4622- symlink_pathname)
4623- os.unlink(symlink_pathname)
4624- finally:
4625- # Finally put the last-session synlink that points to this storage
4626- logger.debug(
4627- _("Linking storage %r to last session"), self.location)
4628- try:
4629- os.symlink(self.location, symlink_pathname)
4630- except OSError as exc:
4631- logger.error(
4632- _("Cannot link %r as %r: %r"),
4633- self.location, symlink_pathname, exc)
4634-
4635 def remove(self):
4636 """
4637 Remove all filesystem entries associated with this instance.
4638@@ -367,164 +271,7 @@ class SessionStorage:
4639
4640 :raises IOError, OSError:
4641 on various problems related to accessing the filesystem
4642-
4643- :raises NotImplementedError:
4644- when openat(2) is not available on this platform. Should never
4645- happen on Linux or Windows where appropriate checks divert to a
4646- correct implementation that is not using them.
4647- """
4648- if sys.platform == 'linux' or sys.platform == 'linux2':
4649- if sys.version_info[0:2] >= (3, 3):
4650- return self._load_checkpoint_unix_py33()
4651- else:
4652- return self._load_checkpoint_unix_py32()
4653- elif sys.platform == 'win32':
4654- return self._load_checkpoint_win32_py33()
4655- raise NotImplementedError(
4656- "platform/python combination is not supported: {} + {}".format(
4657- sys.version, sys.platform))
4658-
4659- def save_checkpoint(self, data):
4660- """
4661- Save checkpoint data to the filesystem.
4662-
4663- The directory associated with this :class:`SessionStorage` must already
4664- exist. Typically the instance should be obtained by calling
4665- :meth:`SessionStorage.create()` which will ensure that this is already
4666- the case.
4667-
4668- :raises TypeError:
4669- if data is not a bytes object.
4670-
4671- :raises LockedStorageError:
4672- if leftovers from previous save_checkpoint() have been detected.
4673- Normally those should never be here but in certain cases that is
4674- possible. Callers might want to call :meth:`break_lock()`
4675- to resolve the problem and try again.
4676-
4677- :raises IOError, OSError:
4678- on various problems related to accessing the filesystem.
4679- Typically permission errors may be reported here.
4680-
4681- :raises NotImplementedError:
4682- when openat(2), renameat(2), unlinkat(2) are not available on this
4683- platform. Should never happen on Linux or Windows where appropriate
4684- checks divert to a correct implementation that is not using them.
4685 """
4686- if sys.platform == 'linux' or sys.platform == 'linux2':
4687- if sys.version_info[0:2] >= (3, 3):
4688- return self._save_checkpoint_unix_py33(data)
4689- else:
4690- return self._save_checkpoint_unix_py32(data)
4691- elif sys.platform == 'win32':
4692- if sys.version_info[0:2] >= (3, 3):
4693- return self._save_checkpoint_win32_py33(data)
4694- raise NotImplementedError(
4695- "platform/python combination is not supported: {} + {}".format(
4696- sys.version, sys.platform))
4697-
4698- def break_lock(self):
4699- """
4700- Forcibly unlock the storage by removing a file created during
4701- atomic filesystem operations of save_checkpoint().
4702-
4703- This method might be useful if save_checkpoint()
4704- raises LockedStorageError. It removes the "next" file that is used
4705- for atomic rename.
4706- """
4707- _next_session_pathname = os.path.join(
4708- self._location, self._SESSION_FILE_NEXT)
4709- logger.debug(
4710- # TRANSLATORS: unlinking as in deleting a file
4711- # Please keep the 'next' string untranslated
4712- _("Forcibly unlinking 'next' file %r"), _next_session_pathname)
4713- os.unlink(_next_session_pathname)
4714-
4715- def _load_checkpoint_win32_py33(self):
4716- logger.debug(_("Loading checkpoint (%s)"), "Windows")
4717- _session_pathname = os.path.join(self._location, self._SESSION_FILE)
4718- try:
4719- # Open the current session file in the location directory
4720- session_fd = os.open(_session_pathname, os.O_RDONLY | os.O_BINARY)
4721- logger.debug(
4722- _("Opened session state file %r as descriptor %d"),
4723- _session_pathname, session_fd)
4724- # Stat the file to know how much to read
4725- session_stat = os.fstat(session_fd)
4726- logger.debug(
4727- # TRANSLATORS: stat is a system call name, don't translate it
4728- _("Stat'ed session state file: %s"), session_stat)
4729- try:
4730- # Read session data
4731- logger.debug(ngettext(
4732- "Reading %d byte of session state",
4733- "Reading %d bytes of session state",
4734- session_stat.st_size), session_stat.st_size)
4735- data = os.read(session_fd, session_stat.st_size)
4736- logger.debug(ngettext(
4737- "Read %d byte of session state",
4738- "Read %d bytes of session state", len(data)), len(data))
4739- if len(data) != session_stat.st_size:
4740- raise IOError(_("partial read?"))
4741- finally:
4742- # Close the session file
4743- logger.debug(_("Closed descriptor %d"), session_fd)
4744- os.close(session_fd)
4745- except IOError as exc:
4746- if exc.errno == errno.ENOENT:
4747- # Treat lack of 'session' file as an empty file
4748- return b''
4749- raise
4750- else:
4751- return data
4752-
4753- def _load_checkpoint_unix_py32(self):
4754- _session_pathname = os.path.join(self._location, self._SESSION_FILE)
4755- # Open the location directory
4756- location_fd = os.open(self._location, os.O_DIRECTORY)
4757- logger.debug(
4758- _("Opened session directory %r as descriptor %d"),
4759- self._location, location_fd)
4760- try:
4761- # Open the current session file in the location directory
4762- session_fd = os.open(_session_pathname, os.O_RDONLY)
4763- logger.debug(
4764- _("Opened session state file %r as descriptor %d"),
4765- _session_pathname, session_fd)
4766- # Stat the file to know how much to read
4767- session_stat = os.fstat(session_fd)
4768- logger.debug(
4769- # TRANSLATORS: stat is a system call name, don't translate it
4770- _("Stat'ed session state file: %s"), session_stat)
4771- try:
4772- # Read session data
4773- logger.debug(ngettext(
4774- "Reading %d byte of session state",
4775- "Reading %d bytes of session state",
4776- session_stat.st_size), session_stat.st_size)
4777- data = os.read(session_fd, session_stat.st_size)
4778- logger.debug(ngettext(
4779- "Read %d byte of session state",
4780- "Read %d bytes of session state", len(data)), len(data))
4781- if len(data) != session_stat.st_size:
4782- raise IOError(_("partial read?"))
4783- finally:
4784- # Close the session file
4785- logger.debug(_("Closed descriptor %d"), session_fd)
4786- os.close(session_fd)
4787- except IOError as exc:
4788- if exc.errno == errno.ENOENT:
4789- # Treat lack of 'session' file as an empty file
4790- return b''
4791- raise
4792- else:
4793- return data
4794- finally:
4795- # Close the location directory
4796- logger.debug(_("Closed descriptor %d"), location_fd)
4797- os.close(location_fd)
4798-
4799- def _load_checkpoint_unix_py33(self):
4800 # Open the location directory
4801 location_fd = os.open(self._location, os.O_DIRECTORY)
4802 try:
4803@@ -552,225 +299,28 @@ class SessionStorage:
4804 # Close the location directory
4805 os.close(location_fd)
4806
4807- def _save_checkpoint_win32_py33(self, data):
4808- # NOTE: this is like _save_checkpoint_py32 but without location_fd
4809- # wich cannot be opened on windows (no os.O_DIRECTORY)
4810- #
4811- # NOTE: The windows version is relatively new and under-tested
4812- # but then again we don't expect to run tests *on* windows, only
4813- # *from* windows so hard data retention requirements are of lesser
4814- # importance.
4815- if not isinstance(data, bytes):
4816- raise TypeError("data must be bytes")
4817- logger.debug(ngettext(
4818- "Saving %d byte of data (%s)",
4819- "Saving %d bytes of data (%s)",
4820- len(data)), len(data), "Windows")
4821- # Helper pathnames, needed because we don't have *at functions
4822- _next_session_pathname = os.path.join(
4823- self._location, self._SESSION_FILE_NEXT)
4824- _session_pathname = os.path.join(self._location, self._SESSION_FILE)
4825- # Open the "next" file in the location_directory
4826- #
4827- # Use "write" + "create" + "exclusive" flags so that no race
4828- # condition is possible.
4829- #
4830- # This will never return -1, it throws IOError when anything is
4831- # wrong. The caller has to catch this.
4832- #
4833- # As a special exception, this code handles EEXISTS and converts
4834- # that to LockedStorageError that can be especially handled by
4835- # some layer above.
4836- try:
4837- next_session_fd = os.open(
4838- _next_session_pathname,
4839- os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_BINARY, 0o644)
4840- except IOError as exc:
4841- if exc.errno == errno.EEXISTS:
4842- raise LockedStorageError()
4843- else:
4844- raise
4845- logger.debug(
4846- _("Opened next session file %s as descriptor %d"),
4847- _next_session_pathname, next_session_fd)
4848- try:
4849- # Write session data to disk
4850- #
4851- # I cannot find conclusive evidence but it seems that
4852- # os.write() handles partial writes internally. In case we do
4853- # get a partial write _or_ we run out of disk space, raise an
4854- # explicit IOError.
4855- num_written = os.write(next_session_fd, data)
4856- logger.debug(ngettext(
4857- "Wrote %d byte of data to descriptor %d",
4858- "Wrote %d bytes of data to descriptor %d",
4859- num_written), num_written, next_session_fd)
4860- if num_written != len(data):
4861- raise IOError(_("partial write?"))
4862- except Exception as exc:
4863- logger.warning(_("Unable to complete write: %s"), exc)
4864- # If anything goes wrong we should unlink the next file.
4865- # TRANSLATORS: unlinking as in deleting a file
4866- logger.warning(_("Unlinking %r: %r"), _next_session_pathname, exc)
4867- os.unlink(_next_session_pathname)
4868- else:
4869- # If the write was successful we must flush kernel buffers.
4870- #
4871- # We want to be sure this data is really on disk by now as we
4872- # may crash the machine soon after this method exits.
4873- logger.debug(
4874- # TRANSLATORS: please don't translate fsync()
4875- _("Calling fsync() on descriptor %d"), next_session_fd)
4876- try:
4877- os.fsync(next_session_fd)
4878- except OSError as exc:
4879- logger.warning(_("Cannot synchronize file %r: %s"),
4880- _next_session_pathname, exc)
4881- finally:
4882- # Close the new session file
4883- logger.debug(_("Closing descriptor %d"), next_session_fd)
4884- os.close(next_session_fd)
4885- # Rename FILE_NEXT over FILE.
4886- logger.debug(_("Renaming %r to %r"),
4887- _next_session_pathname, _session_pathname)
4888- try:
4889- os.replace(_next_session_pathname, _session_pathname)
4890- except Exception as exc:
4891- # Same as above, if we fail we need to unlink the next file
4892- # otherwise any other attempts will not be able to open() it
4893- # with O_EXCL flag.
4894- logger.warning(
4895- _("Unable to rename/overwrite %r to %r: %r"),
4896- _next_session_pathname, _session_pathname, exc)
4897- # TRANSLATORS: unlinking as in deleting a file
4898- logger.warning(_("Unlinking %r"), _next_session_pathname)
4899- os.unlink(_next_session_pathname)
4900-
4901- def _save_checkpoint_unix_py32(self, data):
4902- # NOTE: this is like _save_checkpoint_py33 but without all the
4903- # *at() functions (openat, renameat)
4904- #
4905- # Since we cannot use those functions there is an implicit race
4906- # condition on all open() calls with another process that renames
4907- # any of the directories that are part of the opened path.
4908- #
4909- # I don't think we can really do anything about this in userspace
4910- # so this, python 3.2 specific version, just does the best effort
4911- # implementation. Some of the comments were redacted but
4912- # but keep in mind that the rename race is always there.
4913- if not isinstance(data, bytes):
4914- raise TypeError("data must be bytes")
4915- logger.debug(ngettext(
4916- "Saving %d byte of data (%s)",
4917- "Saving %d bytes of data (%s)",
4918- len(data)), len(data), "UNIX, python 3.2 or older")
4919- # Helper pathnames, needed because we don't have *at functions
4920- _next_session_pathname = os.path.join(
4921- self._location, self._SESSION_FILE_NEXT)
4922- _session_pathname = os.path.join(self._location, self._SESSION_FILE)
4923- # Open the location directory, we need to fsync that later
4924- # XXX: this may fail, maybe we should keep the fd open all the time?
4925- location_fd = os.open(self._location, os.O_DIRECTORY)
4926- logger.debug(
4927- _("Opened %r as descriptor %d"), self._location, location_fd)
4928- try:
4929- # Open the "next" file in the location_directory
4930- #
4931- # Use "write" + "create" + "exclusive" flags so that no race
4932- # condition is possible.
4933- #
4934- # This will never return -1, it throws IOError when anything is
4935- # wrong. The caller has to catch this.
4936- #
4937- # As a special exception, this code handles EEXISTS and converts
4938- # that to LockedStorageError that can be especially handled by
4939- # some layer above.
4940- try:
4941- next_session_fd = os.open(
4942- _next_session_pathname,
4943- os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o644)
4944- except IOError as exc:
4945- if exc.errno == errno.EEXISTS:
4946- raise LockedStorageError()
4947- else:
4948- raise
4949- logger.debug(
4950- _("Opened next session file %s as descriptor %d"),
4951- _next_session_pathname, next_session_fd)
4952- try:
4953- # Write session data to disk
4954- #
4955- # I cannot find conclusive evidence but it seems that
4956- # os.write() handles partial writes internally. In case we do
4957- # get a partial write _or_ we run out of disk space, raise an
4958- # explicit IOError.
4959- num_written = os.write(next_session_fd, data)
4960- logger.debug(ngettext(
4961- "Wrote %d byte of data to descriptor %d",
4962- "Wrote %d bytes of data to descriptor %d",
4963- num_written), num_written, next_session_fd)
4964- if num_written != len(data):
4965- raise IOError(_("partial write?"))
4966- except Exception as exc:
4967- logger.warning(_("Unable to complete write: %r"), exc)
4968- # If anything goes wrong we should unlink the next file.
4969- # TRANSLATORS: unlinking as in deleting a file
4970- logger.warning(_("Unlinking %r"), _next_session_pathname)
4971- os.unlink(_next_session_pathname)
4972- else:
4973- # If the write was successful we must flush kernel buffers.
4974- #
4975- # We want to be sure this data is really on disk by now as we
4976- # may crash the machine soon after this method exits.
4977- logger.debug(
4978- # TRANSLATORS: please don't translate fsync()
4979- _("Calling fsync() on descriptor %d"), next_session_fd)
4980- try:
4981- os.fsync(next_session_fd)
4982- except OSError as exc:
4983- logger.warning(_("Cannot synchronize file %r: %s"),
4984- _next_session_pathname, exc)
4985- finally:
4986- # Close the new session file
4987- logger.debug(_("Closing descriptor %d"), next_session_fd)
4988- os.close(next_session_fd)
4989- # Rename FILE_NEXT over FILE.
4990- logger.debug(_("Renaming %r to %r"),
4991- _next_session_pathname, _session_pathname)
4992- try:
4993- os.rename(_next_session_pathname, _session_pathname)
4994- except Exception as exc:
4995- # Same as above, if we fail we need to unlink the next file
4996- # otherwise any other attempts will not be able to open() it
4997- # with O_EXCL flag.
4998- logger.warning(
4999- _("Unable to rename/overwrite %r to %r: %r"),
5000- _next_session_pathname, _session_pathname, exc)
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches