Merge ~matthiasbaur/cloud-init:feature/puppet-csr-attributes into cloud-init:master

Proposed by Matthias Baur
Status: Merged
Approved by: Ryan Harper
Approved revision: 01e25541305b5c01d3b3fced4364a0e570a52c91
Merge reported by: Server Team CI bot
Merged at revision: not available
Proposed branch: ~matthiasbaur/cloud-init:feature/puppet-csr-attributes
Merge into: cloud-init:master
Diff against target: 158 lines (+63/-5)
2 files modified
cloudinit/config/cc_puppet.py (+29/-5)
tests/unittests/test_handler/test_handler_puppet.py (+34/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Ryan Harper Approve
Dan Watkins Needs Information
Review via email: mp+371997@code.launchpad.net

Commit message

cc_puppet: Implement csr_attributes.yaml support

This change adds two new parameters:
* csr_attributes
* csr_attributes_path

Those parameters allow to configure the content of the
csr_attributes.yaml file.

See https://puppet.com/docs/puppet/latest/config_file_csr_attributes.html

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:5341c52c069b15519079016f95bb2a7ea927f87d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1094/
Executed test runs:
    FAILED: Checkout

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1094//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Hi Matthias,

Thank you for contributing to cloud-init.

To contribute, you must sign the Canonical Contributor License Agreement (CLA) [1].

If you have already signed it as an individual, your Launchpad user will be listed in the contributor-agreement-canonical launchpad group [2]. Unfortunately there is no easy way to check if an organization or company you are doing work for has signed. If you are unsure or have questions, email <email address hidden> or ping smoser in #cloud-init channel via freenode.

For information on how to sign, please see the HACKING document [3].

Thanks again, and please feel free to reach out with any questions.


[1] http://www.canonical.com/contributors
[2] https://launchpad.net/~contributor-agreement-canonical/+members
[3] http://cloudinit.readthedocs.io/en/latest/topics/hacking.html

Revision history for this message
Matthias Baur (matthiasbaur) wrote :

Hey Chad,

thanks for taking a look! I'm not quite sure which tests fail in CI as I can't access the URLs. I tried to run it locally on my Mac which lead to a lot of failing tests even for the master branch.

I checked that the following tests succeed for the files I changed:

* make pep8
* make pyflakes
* make pyflakes3
* nosetests tests/unittests/test_handler/test_handler_puppet.py

Regarding the CLA: I couldn't figure out if the company I work for has already signed the CLA. I've reached out to Scott on IRC who told me that he's not working anymore at Canonical. His suggestion was to contact out to powersj, which I tried but couldn't reach.

I'm out of office for two weeks now, will come back to this afterwards.

Revision history for this message
Ryan Harper (raharper) wrote :

Looks like the CI job had the wrong branch name, I've started up another with the correct parameters.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:5341c52c069b15519079016f95bb2a7ea927f87d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1098/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1098//rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Dan Watkins (oddbloke) wrote :

This is the current error in CI:

======================================================================
FAIL: tests.unittests.test_handler.test_handler_puppet.TestPuppetHandle.test_handler_puppet_writes_csr_attributes_file
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/jenkins/servers/server/workspace/cloud-init-ci/.tox/xenial/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/jenkins/servers/server/workspace/cloud-init-ci/tests/unittests/test_handler/test_handler_puppet.py", line 176, in test_handler_puppet_writes_csr_attributes_file
    self.assertEqual(expected, content)
AssertionError: 'custom_attributes:\n 1.2.840.113549.1.9.7: 342thbjkt82094y[178 chars]2E\n' != 'custom_attributes: {1.2.840.113549.1.9.7: 342thbjkt82094y0u[175 chars]E}\n'
- custom_attributes:
- 1.2.840.113549.1.9.7: 342thbjkt82094y0uthhor289jnqthpc2290
? ^^
+ custom_attributes: {1.2.840.113549.1.9.7: 342thbjkt82094y0uthhor289jnqthpc2290}
? ^^^^^^^^^^^^^^^^^^^^ +
+ extension_requests: {pp_image_name: my_ami_image, pp_preshared_key: 342thbjkt82094y0uthhor289jnqthpc2290,
- extension_requests:
- pp_image_name: my_ami_image
- pp_preshared_key: 342thbjkt82094y0uthhor289jnqthpc2290
- pp_uuid: ED803750-E3C7-44F5-BB08-41A04433FE2E
+ pp_uuid: ED803750-E3C7-44F5-BB08-41A04433FE2E}
? +

----------------------------------------------------------------------
Ran 2042 tests in 12.761s

FAILED (SKIP=51, failures=1)

Revision history for this message
Matthias Baur (matthiasbaur) wrote :

Thanks Dan! :) For reasons unclear to me this error was shown when I ran nosetests...

I've added the missing "default_flow_style=False". Could you please reran the tests?

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:01e25541305b5c01d3b3fced4364a0e570a52c91
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1142/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1142//rebuild

review: Approve (continuous-integration)
Revision history for this message
Dan Watkins (oddbloke) wrote :

LGTM, but marking as Needs Information while we resolve the CLA signing.

review: Needs Information
Revision history for this message
Matthias Baur (matthiasbaur) wrote :

My company now signed the CLA. Is there anything else I need to do?

Revision history for this message
Dan Watkins (oddbloke) wrote :

Hey Matthias, unfortunately for company-level CLA signing I have to bug people internally for confirmation, so that might take a couple of days. We'll get there soon!

Revision history for this message
Ryan Harper (raharper) wrote :

Thanks Matthias,

I've verified the company signing the CLA.

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/config/cc_puppet.py b/cloudinit/config/cc_puppet.py
2index 4190a20..35c3baa 100644
3--- a/cloudinit/config/cc_puppet.py
4+++ b/cloudinit/config/cc_puppet.py
5@@ -24,9 +24,10 @@ module will attempt to start puppet even if no installation was performed.
6 The module also provides keys for configuring the new puppet 4 paths and
7 installing the puppet package from the puppetlabs repositories:
8 https://docs.puppet.com/puppet/4.2/reference/whered_it_go.html
9-The keys are ``package_name``, ``conf_file`` and ``ssl_dir``. If unset, their
10-values will default to ones that work with puppet 3.x and with distributions
11-that ship modified puppet 4.x that uses the old paths.
12+The keys are ``package_name``, ``conf_file``, ``ssl_dir`` and
13+``csr_attributes_path``. If unset, their values will default to
14+ones that work with puppet 3.x and with distributions that ship modified
15+puppet 4.x that uses the old paths.
16
17 Puppet configuration can be specified under the ``conf`` key. The
18 configuration is specified as a dictionary containing high-level ``<section>``
19@@ -40,6 +41,10 @@ If ``ca_cert`` is present, it will not be written to ``puppet.conf``, but
20 instead will be used as the puppermaster certificate. It should be specified
21 in pem format as a multi-line string (using the ``|`` yaml notation).
22
23+Additionally it's possible to create a csr_attributes.yaml for
24+CSR attributes and certificate extension requests.
25+See https://puppet.com/docs/puppet/latest/config_file_csr_attributes.html
26+
27 **Internal name:** ``cc_puppet``
28
29 **Module frequency:** per instance
30@@ -53,6 +58,7 @@ in pem format as a multi-line string (using the ``|`` yaml notation).
31 version: <version>
32 conf_file: '/etc/puppet/puppet.conf'
33 ssl_dir: '/var/lib/puppet/ssl'
34+ csr_attributes_path: '/etc/puppet/csr_attributes.yaml'
35 package_name: 'puppet'
36 conf:
37 agent:
38@@ -62,28 +68,39 @@ in pem format as a multi-line string (using the ``|`` yaml notation).
39 -------BEGIN CERTIFICATE-------
40 <cert data>
41 -------END CERTIFICATE-------
42+ csr_attributes:
43+ custom_attributes:
44+ 1.2.840.113549.1.9.7: 342thbjkt82094y0uthhor289jnqthpc2290
45+ extension_requests:
46+ pp_uuid: ED803750-E3C7-44F5-BB08-41A04433FE2E
47+ pp_image_name: my_ami_image
48+ pp_preshared_key: 342thbjkt82094y0uthhor289jnqthpc2290
49 """
50
51 from six import StringIO
52
53 import os
54 import socket
55+import yaml
56
57 from cloudinit import helpers
58 from cloudinit import util
59
60 PUPPET_CONF_PATH = '/etc/puppet/puppet.conf'
61 PUPPET_SSL_DIR = '/var/lib/puppet/ssl'
62+PUPPET_CSR_ATTRIBUTES_PATH = '/etc/puppet/csr_attributes.yaml'
63 PUPPET_PACKAGE_NAME = 'puppet'
64
65
66 class PuppetConstants(object):
67
68- def __init__(self, puppet_conf_file, puppet_ssl_dir, log):
69+ def __init__(self, puppet_conf_file, puppet_ssl_dir,
70+ csr_attributes_path, log):
71 self.conf_path = puppet_conf_file
72 self.ssl_dir = puppet_ssl_dir
73 self.ssl_cert_dir = os.path.join(puppet_ssl_dir, "certs")
74 self.ssl_cert_path = os.path.join(self.ssl_cert_dir, "ca.pem")
75+ self.csr_attributes_path = csr_attributes_path
76
77
78 def _autostart_puppet(log):
79@@ -118,8 +135,10 @@ def handle(name, cfg, cloud, log, _args):
80 conf_file = util.get_cfg_option_str(
81 puppet_cfg, 'conf_file', PUPPET_CONF_PATH)
82 ssl_dir = util.get_cfg_option_str(puppet_cfg, 'ssl_dir', PUPPET_SSL_DIR)
83+ csr_attributes_path = util.get_cfg_option_str(
84+ puppet_cfg, 'csr_attributes_path', PUPPET_CSR_ATTRIBUTES_PATH)
85
86- p_constants = PuppetConstants(conf_file, ssl_dir, log)
87+ p_constants = PuppetConstants(conf_file, ssl_dir, csr_attributes_path, log)
88 if not install and version:
89 log.warn(("Puppet install set false but version supplied,"
90 " doing nothing."))
91@@ -176,6 +195,11 @@ def handle(name, cfg, cloud, log, _args):
92 % (p_constants.conf_path))
93 util.write_file(p_constants.conf_path, puppet_config.stringify())
94
95+ if 'csr_attributes' in puppet_cfg:
96+ util.write_file(p_constants.csr_attributes_path,
97+ yaml.dump(puppet_cfg['csr_attributes'],
98+ default_flow_style=False))
99+
100 # Set it up so it autostarts
101 _autostart_puppet(log)
102
103diff --git a/tests/unittests/test_handler/test_handler_puppet.py b/tests/unittests/test_handler/test_handler_puppet.py
104index 0b6e3b5..1494177 100644
105--- a/tests/unittests/test_handler/test_handler_puppet.py
106+++ b/tests/unittests/test_handler/test_handler_puppet.py
107@@ -6,6 +6,7 @@ from cloudinit import (distros, helpers, cloud, util)
108 from cloudinit.tests.helpers import CiTestCase, mock
109
110 import logging
111+import textwrap
112
113
114 LOG = logging.getLogger(__name__)
115@@ -64,6 +65,7 @@ class TestPuppetHandle(CiTestCase):
116 super(TestPuppetHandle, self).setUp()
117 self.new_root = self.tmp_dir()
118 self.conf = self.tmp_path('puppet.conf')
119+ self.csr_attributes_path = self.tmp_path('csr_attributes.yaml')
120
121 def _get_cloud(self, distro):
122 paths = helpers.Paths({'templates_dir': self.new_root})
123@@ -140,3 +142,35 @@ class TestPuppetHandle(CiTestCase):
124 content = util.load_file(self.conf)
125 expected = '[agent]\nserver = puppetmaster.example.org\nother = 3\n\n'
126 self.assertEqual(expected, content)
127+
128+ @mock.patch('cloudinit.config.cc_puppet.util.subp')
129+ def test_handler_puppet_writes_csr_attributes_file(self, m_subp, m_auto):
130+ """When csr_attributes is provided
131+ creates file in PUPPET_CSR_ATTRIBUTES_PATH."""
132+ mycloud = self._get_cloud('ubuntu')
133+ mycloud.distro = mock.MagicMock()
134+ cfg = {
135+ 'puppet': {
136+ 'csr_attributes': {
137+ 'custom_attributes': {
138+ '1.2.840.113549.1.9.7': '342thbjkt82094y0ut'
139+ 'hhor289jnqthpc2290'},
140+ 'extension_requests': {
141+ 'pp_uuid': 'ED803750-E3C7-44F5-BB08-41A04433FE2E',
142+ 'pp_image_name': 'my_ami_image',
143+ 'pp_preshared_key': '342thbjkt82094y0uthhor289jnqthpc2290'}
144+ }}}
145+ csr_attributes = 'cloudinit.config.cc_puppet.' \
146+ 'PUPPET_CSR_ATTRIBUTES_PATH'
147+ with mock.patch(csr_attributes, self.csr_attributes_path):
148+ cc_puppet.handle('notimportant', cfg, mycloud, LOG, None)
149+ content = util.load_file(self.csr_attributes_path)
150+ expected = textwrap.dedent("""\
151+ custom_attributes:
152+ 1.2.840.113549.1.9.7: 342thbjkt82094y0uthhor289jnqthpc2290
153+ extension_requests:
154+ pp_image_name: my_ami_image
155+ pp_preshared_key: 342thbjkt82094y0uthhor289jnqthpc2290
156+ pp_uuid: ED803750-E3C7-44F5-BB08-41A04433FE2E
157+ """)
158+ self.assertEqual(expected, content)

Subscribers

People subscribed via source and target branches