Merge lp:~mmoulton/cloud-init/chef into lp:~cloud-init-dev/cloud-init/trunk
- chef
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 446 |
Proposed branch: | lp:~mmoulton/cloud-init/chef |
Merge into: | lp:~cloud-init-dev/cloud-init/trunk |
Diff against target: |
221 lines (+92/-68) 3 files modified
cloudinit/CloudConfig/cc_chef.py (+59/-44) doc/examples/cloud-config-chef.txt (+22/-16) templates/chef_client.rb.tmpl (+11/-8) |
To merge this branch: | bzr merge lp:~mmoulton/cloud-init/chef |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Scott Moser | Pending | ||
Review via email: mp+74900@code.launchpad.net |
Commit message
Description of the change
This branch is based on lp:~avishai-ish-shalom/cloud-init/chef with several bug fixes to support both 'gems' and 'packages' install types.
This branch has been fully tested with the Beta 1 release of Oneiric.
In addition to bug fixes, this branch introduces support for setting the Chef 'node_name' and 'environment' properties.
Scott Moser (smoser) wrote : | # |
- 447. By Mike Moulton
-
Set sane defaults for 'node_name' and 'environment'
Support both 'validation_cert' and 'validation_key' for backwards compatibility
Cleaned up line length
Mike Moulton (mmoulton) wrote : | # |
> A couple things I'd like to see fixed
> - you added 'environment' and 'node_name' configuration variables, but did not
> add defaults. That means a cloud-config that would previously have worked
> will now fail in the rendering of that template. (I think). Please supply
> default values for those.
I have updated the branch to include sane defaults for both node_name and environment.
> - please try to keep lines to less than 80 chars. Some of the lines added are
> well over that.
I've cleaned this up.
> - you've changed 'validation_cert' to 'validation_key', and the example even
> suggest different formatting going into the same file
> (/etc/chef/
In Chef, the client uses a key pair to validate it's request with the server. The key supplied in the 'validation_key' attribute is the private key for this pair. The naming originally implied and showed this value should have been a certificate, leading to confusion.
I have updated the branch to support the original 'validation_cert' and the new 'validation_key' param where 'validation_key' has precidence.
Scott Moser (smoser) wrote : | # |
The changes seem fine. avishai can you test this ?
The one I see that avishai's branch ([1], specifically [2]) has that this one does not have is 'initial_
[1] lp:~avishai-ish-shalom/cloud-init/chef
[2] http://
- 448. By Mike Moulton
-
Bringing in proper json support for firstboot.json from lp:~avishai-ish-shalom/cloud-init/chef
Bringing in 'initial_properties' support from lp:~avishai-ish-shalom/cloud-init/chef
Mike Moulton (mmoulton) wrote : | # |
This branch now models Avishai's branch [1] for both initial_attributes and using json to write the file.
I also fixed a bug where the initial_properties would not be written if the run_list was not specified. This is important as you can push run lists to a node after the fact via the Chef server.
[1] http://
Preview Diff
1 | === modified file 'cloudinit/CloudConfig/cc_chef.py' | |||
2 | --- cloudinit/CloudConfig/cc_chef.py 2011-09-08 23:10:17 +0000 | |||
3 | +++ cloudinit/CloudConfig/cc_chef.py 2011-09-13 00:43:10 +0000 | |||
4 | @@ -1,6 +1,7 @@ | |||
5 | 1 | # vi: ts=4 expandtab | 1 | # vi: ts=4 expandtab |
6 | 2 | # | 2 | # |
7 | 3 | # Author: Avishai Ish-Shalom <avishai@fewbytes.com> | 3 | # Author: Avishai Ish-Shalom <avishai@fewbytes.com> |
8 | 4 | # Author: Mike Moulton <mike@meltmedia.com> | ||
9 | 4 | # | 5 | # |
10 | 5 | # This program is free software: you can redistribute it and/or modify | 6 | # This program is free software: you can redistribute it and/or modify |
11 | 6 | # it under the terms of the GNU General Public License version 3, as | 7 | # it under the terms of the GNU General Public License version 3, as |
12 | @@ -17,6 +18,7 @@ | |||
13 | 17 | import pwd | 18 | import pwd |
14 | 18 | import socket | 19 | import socket |
15 | 19 | import subprocess | 20 | import subprocess |
16 | 21 | import json | ||
17 | 20 | import StringIO | 22 | import StringIO |
18 | 21 | import ConfigParser | 23 | import ConfigParser |
19 | 22 | import cloudinit.CloudConfig as cc | 24 | import cloudinit.CloudConfig as cc |
20 | @@ -31,60 +33,73 @@ | |||
21 | 31 | if not cfg.has_key('chef'): return | 33 | if not cfg.has_key('chef'): return |
22 | 32 | chef_cfg = cfg['chef'] | 34 | chef_cfg = cfg['chef'] |
23 | 33 | 35 | ||
37 | 34 | # Install chef packages from selected source | 36 | # ensure the chef directories we use exist |
38 | 35 | install_type = util.get_cfg_option_str(chef_cfg, "install_type", "packages") | 37 | mkdirs(['/etc/chef', '/var/log/chef', '/var/lib/chef', |
39 | 36 | if not os.path.isfile('/usr/bin/chef-client'): | 38 | '/var/cache/chef', '/var/backups/chef', '/var/run/chef']) |
27 | 37 | if install_type == "gems": | ||
28 | 38 | if chef_cfg.has_key('version'): | ||
29 | 39 | chef_version = chef_cfg['version'] | ||
30 | 40 | else: | ||
31 | 41 | chef_version = None | ||
32 | 42 | install_chef_from_gems( | ||
33 | 43 | util.get_cfg_option_str(chef_cfg, 'ruby_version', '1.8'), | ||
34 | 44 | chef_version) | ||
35 | 45 | else: | ||
36 | 46 | cc.install_packages(('chef',)) | ||
40 | 47 | 39 | ||
45 | 48 | # set the validation cert | 40 | # set the validation key based on the presence of either 'validation_key' |
46 | 49 | if chef_cfg.has_key('validation_cert'): | 41 | # or 'validation_cert'. In the case where both exist, 'validation_key' |
47 | 50 | with open('/etc/chef/validation.pem', 'w') as validation_cert_fh: | 42 | # takes precedence |
48 | 51 | validation_cert_fh.write(chef_cfg['validation_cert']) | 43 | if chef_cfg.has_key('validation_key') or chef_cfg.has_key('validation_cert'): |
49 | 44 | validation_key = util.get_cfg_option_str(chef_cfg, 'validation_key', | ||
50 | 45 | chef_cfg['validation_cert']) | ||
51 | 46 | with open('/etc/chef/validation.pem', 'w') as validation_key_fh: | ||
52 | 47 | validation_key_fh.write(validation_key) | ||
53 | 52 | 48 | ||
54 | 53 | validation_name = chef_cfg.get('validation_name','chef-validator') | 49 | validation_name = chef_cfg.get('validation_name','chef-validator') |
55 | 54 | # create the chef config from template | 50 | # create the chef config from template |
56 | 55 | util.render_to_file('chef_client.rb', '/etc/chef/client.rb', | 51 | util.render_to_file('chef_client.rb', '/etc/chef/client.rb', |
57 | 56 | {'server_url': chef_cfg['server_url'], | 52 | {'server_url': chef_cfg['server_url'], |
58 | 53 | 'node_name': util.get_cfg_option_str(chef_cfg, 'node_name', | ||
59 | 54 | cloud.datasource.get_instance_id()), | ||
60 | 55 | 'environment': util.get_cfg_option_str(chef_cfg, 'environment', | ||
61 | 56 | '_default'), | ||
62 | 57 | 'validation_name': chef_cfg['validation_name']}) | 57 | 'validation_name': chef_cfg['validation_name']}) |
63 | 58 | 58 | ||
64 | 59 | chef_args = ['-d'] | ||
65 | 60 | # set the firstboot json | 59 | # set the firstboot json |
74 | 61 | if chef_cfg.has_key('run_list'): | 60 | with open('/etc/chef/firstboot.json', 'w') as firstboot_json_fh: |
75 | 62 | with open('/etc/chef/firstboot.json', 'w') as firstboot_json_fh: | 61 | initial_json = {} |
76 | 63 | firstboot_json_fh.write("{\n\"run_list\":\n[\n") | 62 | if chef_cfg.has_key('run_list'): |
77 | 64 | firstboot_json_fh.write( | 63 | initial_json['run_list'] = chef_cfg['run_list'] |
78 | 65 | ",\n".join(["\"%s\"" % runlist_item for runlist_item in chef_cfg['run_list']]) | 64 | if chef_cfg.has_key('initial_attributes'): |
79 | 66 | ) | 65 | initial_attributes = chef_cfg['initial_attributes'] |
80 | 67 | firstboot_json_fh.write("]\n\}") | 66 | for k in initial_attributes.keys(): initial_json[k] = initial_attributes[k] |
81 | 68 | chef_args.append('-j /etc/chef/firstboot.json') | 67 | firstboot_json_fh.write(json.dumps(initial_json)) |
82 | 69 | 68 | ||
86 | 70 | # and finally, run chef | 69 | # If chef is not installed, we install chef based on 'install_type' |
87 | 71 | log.debug("running chef-client %s" % chef_args) | 70 | if not os.path.isfile('/usr/bin/chef-client'): |
88 | 72 | subprocess.check_call(['/usr/bin/chef-client'] + chef_args) | 71 | install_type = util.get_cfg_option_str(chef_cfg, 'install_type', 'packages') |
89 | 72 | if install_type == "gems": | ||
90 | 73 | # this will install and run the chef-client from gems | ||
91 | 74 | chef_version = util.get_cfg_option_str(chef_cfg, 'version', None) | ||
92 | 75 | ruby_version = util.get_cfg_option_str(chef_cfg, 'ruby_version', '1.8') | ||
93 | 76 | install_chef_from_gems(ruby_version, chef_version) | ||
94 | 77 | # and finally, run chef-client | ||
95 | 78 | log.debug('running chef-client') | ||
96 | 79 | subprocess.check_call(['/usr/bin/chef-client', '-d', '-i', '1800', '-s', '20']) | ||
97 | 80 | else: | ||
98 | 81 | # this will install and run the chef-client from packages | ||
99 | 82 | cc.install_packages(('chef',)) | ||
100 | 73 | 83 | ||
101 | 74 | def install_chef_from_gems(ruby_version, chef_version = None): | 84 | def install_chef_from_gems(ruby_version, chef_version = None): |
102 | 75 | cc.install_packages(ruby_packages[ruby_version]) | 85 | cc.install_packages(ruby_packages[ruby_version]) |
118 | 76 | gem_bin = get_gem_bin() | 86 | if not os.path.exists('/usr/bin/gem'): |
119 | 77 | if not os.path.exists('/usr/bin/gem'): os.symlink(gem_bin, '/usr/bin/gem') | 87 | os.symlink('/usr/bin/gem%s' % ruby_version, '/usr/bin/gem') |
120 | 78 | chef_version_arg = "" | 88 | if not os.path.exists('/usr/bin/ruby'): |
121 | 79 | if chef_version: chef_version_arg = "-v %s" % chef_version | 89 | os.symlink('/usr/bin/ruby%s' % ruby_version, '/usr/bin/ruby') |
122 | 80 | subprocess.check_call([gem_bin,'install','chef',chef_version_arg, '--no-ri','--no-rdoc','--no-test','-q']) | 90 | if chef_version: |
123 | 81 | os.mkdirs('/etc/chef', '/var/log/chef', '/var/lib/chef', '/var/cache/chef', '/var/backups/chef', '/var/run/chef') | 91 | subprocess.check_call(['/usr/bin/gem','install','chef', |
124 | 82 | os.symlink('/var/lib/gem/%s/bin/chef-client' % ruby_version, '/usr/bin/chef-client') | 92 | '-v %s' % chef_version, '--no-ri', |
125 | 83 | # Ohai ruby plugin breaks if there is no ruby or gem binaries at /usr/bin, so | 93 | '--no-rdoc','--bindir','/usr/bin','-q']) |
126 | 84 | try: os.symlink('/usr/bin/gem%s' % ruby_version, '/usr/bin/gem') | 94 | else: |
127 | 85 | except: pass | 95 | subprocess.check_call(['/usr/bin/gem','install','chef', |
128 | 86 | try: os.symlink('/usr/bin/ruby%s' % ruby_version, '/usr/bin/ruby') | 96 | '--no-ri','--no-rdoc','--bindir', |
129 | 87 | except: pass | 97 | '/usr/bin','-q']) |
130 | 88 | 98 | ||
131 | 89 | def get_gem_bin(): | 99 | def ensure_dir(d): |
132 | 90 | return '/usr/bin/gem%s' % util.get_cfg_option_str(chef_cfg, 'ruby_version', '1.8') | 100 | if not os.path.exists(d): |
133 | 101 | os.makedirs(d) | ||
134 | 102 | |||
135 | 103 | def mkdirs(dirs): | ||
136 | 104 | for d in dirs: | ||
137 | 105 | ensure_dir(d) | ||
138 | 91 | 106 | ||
139 | === modified file 'doc/examples/cloud-config-chef.txt' | |||
140 | --- doc/examples/cloud-config-chef.txt 2011-04-21 14:57:54 +0000 | |||
141 | +++ doc/examples/cloud-config-chef.txt 2011-09-13 00:43:10 +0000 | |||
142 | @@ -9,30 +9,36 @@ | |||
143 | 9 | apt_mirror: http://apt.opscode.com/ | 9 | apt_mirror: http://apt.opscode.com/ |
144 | 10 | 10 | ||
145 | 11 | chef: | 11 | chef: |
147 | 12 | # If you want to install from rubygems: | 12 | |
148 | 13 | # Valid values are 'gems' and 'packages' | ||
149 | 13 | install_type: "gems" | 14 | install_type: "gems" |
150 | 14 | 15 | ||
151 | 15 | # Chef settings | 16 | # Chef settings |
152 | 16 | server_url: "https://chef.yourorg.com:4000" | 17 | server_url: "https://chef.yourorg.com:4000" |
153 | 17 | 18 | ||
154 | 19 | # Node Name | ||
155 | 20 | # Defaults to the instance-id if not present | ||
156 | 21 | node_name: "your-node-name" | ||
157 | 22 | |||
158 | 23 | # Environment | ||
159 | 24 | # Defaults to '_default' if not present | ||
160 | 25 | environment: "production" | ||
161 | 26 | |||
162 | 18 | # Default validation name is chef-validator | 27 | # Default validation name is chef-validator |
163 | 19 | validation_name: "yourorg-validator" | 28 | validation_name: "yourorg-validator" |
179 | 20 | validation_cert: | | 29 | validation_key: | |
180 | 21 | -----BEGIN CERTIFICATE----- | 30 | -----BEGIN RSA PRIVATE KEY----- |
181 | 22 | MIICCTCCAXKgAwIBAgIBATANBgkqhkiG9w0BAQUFADANMQswCQYDVQQDDAJjYTAe | 31 | YOUR-ORGS-VALIDATION-KEY-HERE |
182 | 23 | Fw0xMDAyMTUxNzI5MjFaFw0xNTAyMTQxNzI5MjFaMA0xCzAJBgNVBAMMAmNhMIGf | 32 | -----END RSA PRIVATE KEY----- |
183 | 24 | MA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCu7Q40sm47/E1Pf+r8AYb/V/FWGPgc | 33 | |
169 | 25 | b014OmNoX7dgCxTDvps/h8Vw555PdAFsW5+QhsGr31IJNI3kSYprFQcYf7A8tNWu | ||
170 | 26 | 1MASW2CfaEiOEi9F1R3R4Qlz4ix+iNoHiUDTjazw/tZwEdxaQXQVLwgTGRwVa+aA | ||
171 | 27 | qbutJKi93MILLwIDAQABo3kwdzA4BglghkgBhvhCAQ0EKxYpUHVwcGV0IFJ1Ynkv | ||
172 | 28 | T3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwDwYDVR0TAQH/BAUwAwEB/zAd | ||
173 | 29 | BgNVHQ4EFgQUu4+jHB+GYE5Vxo+ol1OAhevspjAwCwYDVR0PBAQDAgEGMA0GCSqG | ||
174 | 30 | SIb3DQEBBQUAA4GBAH/rxlUIjwNb3n7TXJcDJ6MMHUlwjr03BDJXKb34Ulndkpaf | ||
175 | 31 | +GAlzPXWa7bO908M9I8RnPfvtKnteLbvgTK+h+zX1XCty+S2EQWk29i2AdoqOTxb | ||
176 | 32 | hppiGMp0tT5Havu4aceCXiy2crVcudj3NFciy8X66SoECemW9UYDCb9T5D0d | ||
177 | 33 | -----END CERTIFICATE----- | ||
178 | 34 | |||
184 | 35 | # A run list for a first boot json | 34 | # A run list for a first boot json |
185 | 36 | run_list: | 35 | run_list: |
186 | 37 | - "recipe[apache2]" | 36 | - "recipe[apache2]" |
187 | 38 | - "role[db]" | 37 | - "role[db]" |
188 | 38 | |||
189 | 39 | # Specify a list of initial attributes used by the cookbooks | ||
190 | 40 | initial_attributes: | ||
191 | 41 | apache: | ||
192 | 42 | prefork: | ||
193 | 43 | maxclients: 100 | ||
194 | 44 | keepalive: "off" | ||
195 | 39 | 45 | ||
196 | === modified file 'templates/chef_client.rb.tmpl' | |||
197 | --- templates/chef_client.rb.tmpl 2011-04-21 14:57:54 +0000 | |||
198 | +++ templates/chef_client.rb.tmpl 2011-09-13 00:43:10 +0000 | |||
199 | @@ -1,12 +1,15 @@ | |||
203 | 1 | log_level :info | 1 | log_level :info |
204 | 2 | log_location "/var/log/chef/client.log" | 2 | log_location "/var/log/chef/client.log" |
205 | 3 | ssl_verify_mode :verify_none | 3 | ssl_verify_mode :verify_none |
206 | 4 | validation_client_name "$validation_name" | 4 | validation_client_name "$validation_name" |
207 | 5 | validation_key "/etc/chef/validation.pem" | 5 | validation_key "/etc/chef/validation.pem" |
213 | 6 | client_key "/etc/chef/client.pem" | 6 | client_key "/etc/chef/client.pem" |
214 | 7 | chef_server_url "$server_url" | 7 | chef_server_url "$server_url" |
215 | 8 | file_cache_path "/var/cache/chef" | 8 | environment "$environment" |
216 | 9 | file_backup_path "/var/backups/chef" | 9 | node_name "$node_name" |
217 | 10 | pid_file "/var/run/chef/client.pid" | 10 | json_attribs "/etc/chef/firstboot.json" |
218 | 11 | file_cache_path "/var/cache/chef" | ||
219 | 12 | file_backup_path "/var/backups/chef" | ||
220 | 13 | pid_file "/var/run/chef/client.pid" | ||
221 | 11 | Chef::Log::Formatter.show_time = true | 14 | Chef::Log::Formatter.show_time = true |
222 | 12 | 15 |
A couple things I'd like to see fixed validation. pem). Why is that?
- you added 'environment' and 'node_name' configuration variables, but did not add defaults. That means a cloud-config that would previously have worked will now fail in the rendering of that template. (I think). Please supply default values for those.
- please try to keep lines to less than 80 chars. Some of the lines added are well over that.
- you've changed 'validation_cert' to 'validation_key', and the example even suggest different formatting going into the same file (/etc/chef/