Merge lp:~bcsaller/pyjuju/lxc-omega into lp:pyjuju
- lxc-omega
- Merge into trunk
Status: | Merged | ||||
---|---|---|---|---|---|
Merge reported by: | Benjamin Saller | ||||
Merged at revision: | not available | ||||
Proposed branch: | lp:~bcsaller/pyjuju/lxc-omega | ||||
Merge into: | lp:pyjuju | ||||
Prerequisite: | lp:~hazmat/pyjuju/local-provider-config | ||||
Diff against target: |
643 lines (+152/-115) 16 files modified
examples/oneiric/mysql/hooks/db-relation-joined (+1/-5) examples/oneiric/wordpress/hooks/config-changed (+1/-1) examples/oneiric/wordpress/hooks/db-relation-changed (+1/-1) examples/oneiric/wordpress/hooks/install (+1/-1) juju/agents/machine.py (+2/-1) juju/control/add_unit.py (+1/-0) juju/lib/lxc/data/juju-create (+1/-0) juju/machine/tests/test_unit_deployment.py (+54/-31) juju/machine/unit.py (+49/-23) juju/providers/lxc/__init__.py (+13/-4) juju/providers/lxc/agent.py (+11/-1) juju/providers/lxc/container.py (+0/-31) juju/providers/lxc/tests/test_container.py (+7/-7) juju/providers/lxc/tests/test_provider.py (+7/-7) misc/bash_completion.d/juju (+2/-2) setup.py (+1/-0) |
||||
To merge this branch: | bzr merge lp:~bcsaller/pyjuju/lxc-omega | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Gustavo Niemeyer | Approve | ||
Kapil Thangavelu (community) | Approve | ||
Review via email: mp+77032@code.launchpad.net |
This proposal supersedes a proposal from 2011-09-25.
Commit message
Description of the change
LXC driven local development story
This branch includes the merge of many preceding branches related to the lxc local dev story. In its current state it can be used to do local lxc deployemnt and testing.
Add the following to your ~/.juju/
juju: environments
default: laptop
laptop
type: lxc
dmin-secret: whoknows
data-dir: /tmp
I recommend adding dmsmasq to your /etc/resolv.conf with:
nameserver 192.168.122.1
However a subsequent branch makes status and ssh find the proper VM for you.
Machines are named via a pattern <username>-<env name>-<
So for me a wordpress deployment from the local repo would result in a machine at
Gustavo Niemeyer (niemeyer) wrote : Posted in a previous version of this proposal | # |
Gustavo Niemeyer (niemeyer) wrote : | # |
There's still no information on how to review this branch without looking at
changes already reviewed in previous branches.
Please do mention what are the pre-requisite branches in the summary, when it
contains multiple branches.
One detail I could already figure on it:
[1]
+@contextlib.
+def lock_manager(path):
+ """Use a file to represent a lock.
(...)
+ # This will raise if the file exists
+ fd = os.open(path, os.O_CREAT | os.O_RDWR | os.O_EXCL)
Please use the existing filesystem lock in Twisted. This version
will explode badly when the second process tries to get into the
mutex region.
Gustavo Niemeyer (niemeyer) wrote : | # |
Kapil mentioned the file-lock branch that is missing from the kanban. I've moved this review there.
Benjamin Saller (bcsaller) wrote : | # |
Uses twisted filelock impl
Uses latest lxc-lib
Examples based around IP address rather than hostname to avoid need to include resolv.conf change on host to address machines (the containers can address each other fine either way)
Kapil Thangavelu (hazmat) wrote : | # |
Looks good to me, some minors
[0] Please use self.juju_home as the base for the lock
762 + lockfile = FilesystemLock(
763 + "/tmp/", "%s.lock" % container_
[1] I'd prefer the master template to be namespace'd to the environment, it makes its cleanup automatic (if it gets broken), and allows divergence in series across environments.
[2] There's a few minor style violations, mostly of the over 80 col variety, ie.
+ if self._public_key:
+ args.insert(1, "JUJU_PUBLIC_
Gustavo Niemeyer (niemeyer) wrote : | # |
There are a few details, but this looks generally nice, thanks.
[2]
+ self.master_
+ container_
+ origin="ppa",
Shouldn't this be dependent on the source of juju itself?
Please check out the env-origin branch that hopefully Jim is landing tomorrow.
Should be trivial to hook on it.
[3]
-hostname=`curl http://
-
+hostname=`ip -f inet addr |grep 192.168| grep -v '192\.168\.1\.' | awk '{print $2;}'|cut -f 1 -d '/'`
Please drop the example changes for now. It break the examples in EC2, and consequently our waterfall.
[4]
I haven't followed carefully, but my gut feeling is that the branch is
a bit shallow on testing. Can you please verify if the following logic
is well covered:
- get_master_template
- start using get_master_template
- JUJU_PUBLIC_KEY handling in agent.py
- etc
[5]
+ log.debug("Created master container %s" %
+ container_
Indentation is off.
[6]
+ try:
+ lockfile.lock()
The locking should be outside the try.. if it fails for whatever
reason, it shouldn't be unlocked.
[7]
Considering that this is using Twisted, shouldn't the lock and unlock
be yielding? I suspect they might be a NOOP at the moment.
[8]
+ if public_
+ public_key = public_key[1:-1]
Please strip it rather than assuming the position blindly. This logic
breaks if e.g. there are spaces at the end of the line.
[9]
+ lockfile = FilesystemLock(
+ "/tmp/", "%s.lock" % container_
That's a security bug. One should never write any files to a publicly
writable location with a known name.
[10]
After you do these changes, can you please ask for another review from
Kapil. I'll be traveling tomorrow, but if he's happy with the branch
I'm +1 on it too. Thank you!
Benjamin Saller (bcsaller) wrote : | # |
> [2]
>
> + self.master_
> + container_
> + origin="ppa",
>
> Shouldn't this be dependent on the source of juju itself?
>
> Please check out the env-origin branch that hopefully Jim is landing tomorrow.
> Should be trivial to hook on it.
I'd like to push this to a later branch, for today this will work with
the ppa and enables local development.
>
> [3]
>
>
> -hostname=`curl http://
> -
> +hostname=`ip -f inet addr |grep 192.168| grep -v '192\.168\.1\.' | awk '{print $2;}'|cut -f 1 -d '/'`
>
> Please drop the example changes for now. It break the examples in EC2, and consequently our waterfall.
>
Changed.
>
> [4]
>
> I haven't followed carefully, but my gut feeling is that the branch is
> a bit shallow on testing. Can you please verify if the following logic
> is well covered:
>
> - get_master_template
> - start using get_master_template
> - JUJU_PUBLIC_KEY handling in agent.py
> - etc
>
Ådded tests around _get_container which encapsulates
get_master_
marks those as internal and the interface made them a bit simpler to
mock in the tests.
[6,7,9] I've removed the lock file, it was a theoretical issue and
wouldn't prevent manual lxc commands from interfering anyway. Instead
we better namespace the template name and check to see if containers
are built to attempt to avoid other conflicts. This new behavior
should aid in restarts of the agent down the line.
> + if public_
> + public_key = public_key[1:-1]
>
> Please strip it rather than assuming the position blindly. This logic
> breaks if e.g. there are spaces at the end of the line.
Done.
>
>
>
> [10]
>
> After you do these changes, can you please ask for another review from
> Kapil. I'll be traveling tomorrow, but if he's happy with the branch
> I'm +1 on it too. Thank you!
Kapil Thangavelu (hazmat) wrote : | # |
post commit review..
[0]
if not master_
yield self.master_
That's a bug, there is no instance attribute self.master_
[1] Lots of over 80 lines, diff to fix (also includes fix for 0)
http://
[2] test_containers should be subsumed along with the implmeentation into the
lib/lxc package.
Preview Diff
1 | === modified file 'examples/oneiric/mysql/hooks/db-relation-joined' |
2 | --- examples/oneiric/mysql/hooks/db-relation-joined 2011-09-15 18:56:08 +0000 |
3 | +++ examples/oneiric/mysql/hooks/db-relation-joined 2011-09-29 06:40:31 +0000 |
4 | @@ -2,11 +2,7 @@ |
5 | |
6 | set -eu # -x for verbose logging to juju debug-log |
7 | |
8 | -# XXX The local hostname should be provided by juju itself in a |
9 | -# future release, so as to remove the dependency of the charm on the |
10 | -# cloud provider (EC2 in this case) |
11 | -hostname=`curl http://169.254.169.254/latest/meta-data/local-hostname` |
12 | - |
13 | +hostname=`hostname` |
14 | # Get the mysql password that was generated by the install hook |
15 | password=`cat /var/lib/juju/mysql.passwd` |
16 | |
17 | |
18 | === modified file 'examples/oneiric/wordpress/hooks/config-changed' |
19 | --- examples/oneiric/wordpress/hooks/config-changed 2011-09-15 18:56:08 +0000 |
20 | +++ examples/oneiric/wordpress/hooks/config-changed 2011-09-29 06:40:31 +0000 |
21 | @@ -1,6 +1,6 @@ |
22 | #!/bin/bash |
23 | |
24 | -hostname=`curl -s http://169.254.169.254/latest/meta-data/public-hostname` |
25 | +hostname=`hostname` |
26 | plugin_dir="/var/www/$hostname/wp-content/plugins" |
27 | plugin_url=`config-get plugin` |
28 | |
29 | |
30 | === modified file 'examples/oneiric/wordpress/hooks/db-relation-changed' |
31 | --- examples/oneiric/wordpress/hooks/db-relation-changed 2011-09-15 18:56:08 +0000 |
32 | +++ examples/oneiric/wordpress/hooks/db-relation-changed 2011-09-29 06:40:31 +0000 |
33 | @@ -2,12 +2,12 @@ |
34 | |
35 | set -eu # -x for verbose logging to juju debug-log |
36 | |
37 | +hostname=`hostname` |
38 | UPLOAD_PATH="/var/www/wp-uploads" |
39 | |
40 | # XXX The public hostname should be provided by juju itself in a |
41 | # future release, so as to remove the dependency of the charm on the |
42 | # cloud provider (EC2 in this case) |
43 | -hostname=`curl http://169.254.169.254/latest/meta-data/public-hostname` |
44 | juju-log "Retrieved hostname: $hostname" |
45 | |
46 | # Check we haven't already been setup. |
47 | |
48 | === modified file 'examples/oneiric/wordpress/hooks/install' |
49 | --- examples/oneiric/wordpress/hooks/install 2011-09-15 18:56:08 +0000 |
50 | +++ examples/oneiric/wordpress/hooks/install 2011-09-29 06:40:31 +0000 |
51 | @@ -3,4 +3,4 @@ |
52 | juju-log "Installing wordpress, pwgen via apt-get" |
53 | set -eu # -x for verbose logging to juju debug-log |
54 | |
55 | -apt-get -y install wordpress pwgen unzip |
56 | +apt-get -y install wordpress pwgen unzip curl |
57 | |
58 | === modified file 'juju/agents/machine.py' |
59 | --- juju/agents/machine.py 2011-09-28 09:48:30 +0000 |
60 | +++ juju/agents/machine.py 2011-09-29 06:40:31 +0000 |
61 | @@ -96,7 +96,7 @@ |
62 | same charm. |
63 | """ |
64 | |
65 | - log.debug("Downloading charm %s...", charm_state.id) |
66 | + log.debug("Downloading charm %s to %s", charm_state.id, self.charms_directory) |
67 | return download_charm( |
68 | self.client, charm_state.id, self.charms_directory) |
69 | |
70 | @@ -151,6 +151,7 @@ |
71 | |
72 | running = yield deployment.is_running() |
73 | if not running: |
74 | + log.debug("Starting service unit %s", service_unit_name) |
75 | yield deployment.start( |
76 | self.get_machine_id(), self.client.servers, bundle) |
77 | log.info("Started service unit %s", service_unit_name) |
78 | |
79 | === modified file 'juju/control/add_unit.py' |
80 | --- juju/control/add_unit.py 2011-09-27 02:14:56 +0000 |
81 | +++ juju/control/add_unit.py 2011-09-29 06:40:31 +0000 |
82 | @@ -3,6 +3,7 @@ |
83 | from twisted.internet.defer import inlineCallbacks |
84 | |
85 | from juju.control.utils import get_environment |
86 | +from juju.state.machine import MachineStateManager |
87 | from juju.state.placement import place_unit |
88 | from juju.state.service import ServiceStateManager |
89 | |
90 | |
91 | === modified file 'juju/lib/lxc/data/juju-create' |
92 | --- juju/lib/lxc/data/juju-create 2011-09-27 03:42:49 +0000 |
93 | +++ juju/lib/lxc/data/juju-create 2011-09-29 06:40:31 +0000 |
94 | @@ -19,6 +19,7 @@ |
95 | |
96 | # By listing the container name first we can expect hostname and |
97 | # hostname -f to return the routable name |
98 | + # note that lxc-clone will overwrite this |
99 | cat <<EOF >/etc/hosts |
100 | 127.0.0.1 $JUJU_CONTAINER_NAME localhost |
101 | EOF |
102 | |
103 | === modified file 'juju/machine/tests/test_unit_deployment.py' |
104 | --- juju/machine/tests/test_unit_deployment.py 2011-09-28 09:48:30 +0000 |
105 | +++ juju/machine/tests/test_unit_deployment.py 2011-09-29 06:40:31 +0000 |
106 | @@ -3,18 +3,17 @@ |
107 | """ |
108 | import logging |
109 | import os |
110 | -import subprocess |
111 | import sys |
112 | |
113 | from twisted.internet.protocol import ProcessProtocol |
114 | from twisted.internet.defer import inlineCallbacks, succeed |
115 | |
116 | import juju |
117 | - |
118 | from juju.charm import get_charm_from_path |
119 | from juju.charm.tests import local_charm_id |
120 | from juju.charm.tests.test_repository import RepositoryTestBase |
121 | -from juju.lib.mocker import MATCH |
122 | +from juju.lib.lxc import LXCContainer |
123 | +from juju.lib.mocker import MATCH, ANY |
124 | from juju.lib.twistutils import get_module_directory |
125 | from juju.machine.unit import UnitMachineDeployment, UnitContainerDeployment |
126 | from juju.machine.errors import UnitDeploymentError |
127 | @@ -328,9 +327,11 @@ |
128 | upstart_job_sample = '''\ |
129 | description "Unit agent for riak/0" |
130 | author "Juju Team <juju@lists.canonical.com>" |
131 | -start on (runlevel [2345] and stopped networking RESULT=ok) |
132 | +start on start on filesystem or runlevel [2345] |
133 | stop on runlevel [!2345] |
134 | + |
135 | respawn |
136 | + |
137 | env JUJU_MACHINE_ID="0" |
138 | env JUJU_HOME="/var/lib/juju" |
139 | env JUJU_ZOOKEEPER="127.0.1.1:2181" |
140 | @@ -368,33 +369,17 @@ |
141 | def test_get_upstart_job(self): |
142 | upstart_job = self.unit_deploy.get_upstart_unit_job( |
143 | 0, "127.0.1.1:2181") |
144 | - self.assertEqual(self.get_normalized(upstart_job), upstart_job_sample) |
145 | - |
146 | - @inlineCallbacks |
147 | - def test_is_running(self): |
148 | - name = self.unit_deploy.container_name |
149 | - mock_lxc_ls = self.mocker.mock() |
150 | - |
151 | - self.patch(subprocess, "check_output", mock_lxc_ls) |
152 | - mock_lxc_ls(["lxc-ls"]) |
153 | - self.mocker.result("%s\n%s" % (name, name)) |
154 | - |
155 | - mock_lxc_ls(["lxc-ls"]) |
156 | - self.mocker.result("%s\n" % name) |
157 | - |
158 | - mock_lxc_ls(["lxc-ls"]) |
159 | - self.mocker.result("abc-0\n") |
160 | - self.mocker.replay() |
161 | - |
162 | - self.assertTrue((yield self.unit_deploy.is_running())) |
163 | - self.assertFalse((yield self.unit_deploy.is_running())) |
164 | - self.assertFalse((yield self.unit_deploy.is_running())) |
165 | + job = self.get_normalized(upstart_job) |
166 | + self.assertIn('JUJU_ZOOKEEPER="127.0.1.1:2181"', job) |
167 | + self.assertIn('JUJU_MACHINE_ID="0"', job) |
168 | + self.assertIn('JUJU_UNIT_NAME="riak/0"', job) |
169 | |
170 | @inlineCallbacks |
171 | def test_destroy(self): |
172 | mock_container = self.mocker.patch(self.unit_deploy.container) |
173 | mock_container.destroy() |
174 | self.mocker.replay() |
175 | + |
176 | yield self.unit_deploy.destroy() |
177 | |
178 | output = self.output.getvalue() |
179 | @@ -403,19 +388,36 @@ |
180 | |
181 | @inlineCallbacks |
182 | def test_start(self): |
183 | - self.unit_deploy.directory = rootfs = self.makeDir() |
184 | + container = LXCContainer(self.unit_name, None, None, None) |
185 | + rootfs = self.makeDir() |
186 | + env = dict(os.environ) |
187 | + env["JUJU_PUBLIC_KEY"] = "dsa ..." |
188 | + self.change_environment(**env) |
189 | + |
190 | + mock_deploy = self.mocker.patch(self.unit_deploy) |
191 | + # this minimally validates that we are also called with the |
192 | + # expect public key |
193 | + mock_deploy._get_container(ANY, ANY, ANY, env["JUJU_PUBLIC_KEY"]) |
194 | + self.mocker.result((container, rootfs)) |
195 | + |
196 | + mock_container = self.mocker.patch(container) |
197 | + mock_container.run() |
198 | + |
199 | + self.mocker.replay() |
200 | + |
201 | + self.unit_deploy.directory = rootfs |
202 | os.makedirs(os.path.join(rootfs, "etc", "init")) |
203 | - mock_container = self.mocker.patch(self.unit_deploy.container) |
204 | - mock_container.run() |
205 | - self.mocker.replay() |
206 | |
207 | yield self.unit_deploy.start("0", "127.0.1.1:2181", self.bundle) |
208 | |
209 | # Verify the upstart job |
210 | - upstart_agent_name = "%s-unit-agent" % self.unit_name.replace("/", "-") |
211 | + upstart_agent_name = "%s-unit-agent.conf" % self.unit_name.replace("/", "-") |
212 | content = open( |
213 | os.path.join(rootfs, "etc", "init", upstart_agent_name)).read() |
214 | - self.assertEqual(self.get_normalized(content), upstart_job_sample) |
215 | + job = self.get_normalized(content) |
216 | + self.assertIn('JUJU_ZOOKEEPER="127.0.1.1:2181"', job) |
217 | + self.assertIn('JUJU_MACHINE_ID="0"', job) |
218 | + self.assertIn('JUJU_UNIT_NAME="riak/0"', job) |
219 | |
220 | # Verify the symlink exists |
221 | self.assertTrue(os.path.lexists(os.path.join( |
222 | @@ -432,3 +434,24 @@ |
223 | self.assertIn("Charm extracted into container", output) |
224 | self.assertIn("Started container for %s" % self.unit_deploy.unit_name, |
225 | output) |
226 | + |
227 | + @inlineCallbacks |
228 | + def test_get_container(self): |
229 | + rootfs = self.makeDir() |
230 | + container = LXCContainer(self.unit_name, None, None, None) |
231 | + |
232 | + mock_deploy = self.mocker.patch(self.unit_deploy) |
233 | + mock_deploy._get_master_template(ANY, ANY, ANY) |
234 | + self.mocker.result(container) |
235 | + |
236 | + mock_container = self.mocker.patch(container) |
237 | + mock_container.clone(ANY) |
238 | + self.mocker.result(container) |
239 | + |
240 | + self.mocker.replay() |
241 | + |
242 | + container, rootfs = yield self.unit_deploy._get_container("0", "127.0.0.1:2181", None, "dsa...") |
243 | + |
244 | + output = self.output.getvalue() |
245 | + self.assertIn("Container created for %s" % self.unit_deploy.unit_name , output) |
246 | + |
247 | |
248 | === modified file 'juju/machine/unit.py' |
249 | --- juju/machine/unit.py 2011-09-23 20:32:51 +0000 |
250 | +++ juju/machine/unit.py 2011-09-29 06:40:31 +0000 |
251 | @@ -13,8 +13,7 @@ |
252 | |
253 | from juju.charm.bundle import CharmBundle |
254 | from juju.lib.twistutils import get_module_directory |
255 | -from juju.lib.lxc import LXCContainer |
256 | -from juju.providers.lxc.container import get_containers |
257 | +from juju.lib.lxc import LXCContainer, get_containers, LXCError |
258 | |
259 | from .errors import UnitDeploymentError |
260 | |
261 | @@ -155,7 +154,7 @@ |
262 | description "Unit agent for %(JUJU_UNIT_NAME)s" |
263 | author "Juju Team <juju@lists.canonical.com>" |
264 | |
265 | -start on (runlevel [2345] and stopped networking RESULT=ok) |
266 | +start on start on filesystem or runlevel [2345] |
267 | stop on runlevel [!2345] |
268 | |
269 | respawn |
270 | @@ -191,15 +190,8 @@ |
271 | self._unit_namespace = os.environ.get("JUJU_UNIT_NS") |
272 | assert self._unit_namespace is not None, "Required unit ns not found" |
273 | |
274 | - self.container = LXCContainer( |
275 | - self.container_name, |
276 | - {"JUJU_ORIGIN": "ppa", |
277 | - "JUJU_SOURCE": "ppa:juju/ppa"}, |
278 | - #console_log=os.path.join(self.directory, "console.log"), |
279 | - #debug_log=os.path.join(self.directory, "container.log")) |
280 | - ) |
281 | - self.directory = self.container.rootfs |
282 | self.pid_file = None |
283 | + self.container = LXCContainer(self.container_name, None, None, None) |
284 | |
285 | @property |
286 | def container_name(self): |
287 | @@ -251,32 +243,63 @@ |
288 | os.makedirs(host_unit_dir) |
289 | |
290 | @inlineCallbacks |
291 | + def _get_master_template(self, machine_id, zookeeper_hosts, public_key): |
292 | + container_template_name = "%s-%s-template" % (self._unit_namespace, machine_id) |
293 | + |
294 | + master_template = LXCContainer(container_template_name, origin="ppa", public_key=public_key) |
295 | + |
296 | + if not master_template.is_constructed(): |
297 | + yield self.master_template.create() |
298 | + log.debug("Created master container %s" % container_template_name) |
299 | + |
300 | + # it wasn't constructed and we couldn't construct it |
301 | + if not master_template.is_constructed(): |
302 | + raise LXCError("Unable to create master container") |
303 | + |
304 | + returnValue(master_template) |
305 | + |
306 | + @inlineCallbacks |
307 | + def _get_container(self, machine_id, zookeeper_hosts, bundle, public_key): |
308 | + master_template = yield self._get_master_template(machine_id, zookeeper_hosts, public_key) |
309 | + log.info( |
310 | + "Creating container %s...", os.path.basename(self.directory)) |
311 | + |
312 | + container = yield master_template.clone(self.container_name) |
313 | + directory = container.rootfs |
314 | + log.info("Container created for %s" % self.unit_name) |
315 | + returnValue((container, directory)) |
316 | + |
317 | + @inlineCallbacks |
318 | def start(self, machine_id, zookeeper_hosts, bundle): |
319 | """Start the unit. |
320 | |
321 | Creates and starts an lxc container for the unit. |
322 | """ |
323 | - if not os.path.exists(self.directory): |
324 | - self.container.configuration["JUJU_ZOOKEEPER"] = zookeeper_hosts |
325 | - log.info( |
326 | - "Creating container %s...", os.path.basename(self.directory)) |
327 | - yield self.container.create() |
328 | - log.info("Container created for %s" % self.unit_name) |
329 | + # remove any quoting around the key |
330 | + public_key = os.environ.get("JUJU_PUBLIC_KEY", "") |
331 | + public_key = public_key.strip("'\"") |
332 | |
333 | + # Build a template container that can be cloned in deploy |
334 | + # we leave the loosely initialized self.container in place for |
335 | + # the class as thats all we need for methods other than start. |
336 | + container, directory = yield self._get_container(machine_id, |
337 | + zookeeper_hosts, |
338 | + bundle, |
339 | + public_key) |
340 | # Create state directories for unit in the container |
341 | self.setup_directories() |
342 | |
343 | # Extract the charm bundle |
344 | charm_path = os.path.join( |
345 | - self.directory, "var", "lib", "juju", "units", |
346 | + directory, "var", "lib", "juju", "units", |
347 | self.unit_path_name, "charm") |
348 | bundle.extract_to(charm_path) |
349 | log.debug("Charm extracted into container") |
350 | |
351 | # Write upstart file for the agent into the container |
352 | upstart_path = os.path.join( |
353 | - self.directory, "etc", "init", |
354 | - "%s-unit-agent" % self.unit_path_name) |
355 | + directory, "etc", "init", |
356 | + "%s-unit-agent.conf" % self.unit_path_name) |
357 | with open(upstart_path, "w") as fh: |
358 | fh.write(self.get_upstart_unit_job(machine_id, zookeeper_hosts)) |
359 | |
360 | @@ -285,12 +308,12 @@ |
361 | self.juju_home, "units", self.unit_path_name, "unit.log") |
362 | if not os.path.lexists(unit_log_path_host): |
363 | os.symlink( |
364 | - os.path.join(self.directory, "var", "log", "juju", |
365 | + os.path.join(directory, "var", "log", "juju", |
366 | "unit-%s.log" % self.unit_path_name), |
367 | unit_log_path_host) |
368 | |
369 | log.debug("Starting container...") |
370 | - yield self.container.run() |
371 | + yield container.run() |
372 | log.info("Started container for %s" % self.unit_name) |
373 | |
374 | @inlineCallbacks |
375 | @@ -308,7 +331,10 @@ |
376 | # TODO: container running may not imply agent running. the |
377 | # pid file has the pid from the container, we need a container |
378 | # pid -> host pid mapping to query status from the machine agent. |
379 | - # alternatively querying zookeeper for the unit agent presence node. |
380 | + # alternatively querying zookeeper for the unit agent presence |
381 | + # node. |
382 | + if not self.container: |
383 | + returnValue(False) |
384 | container_map = yield get_containers( |
385 | prefix=self.container.container_name) |
386 | returnValue(container_map.get(self.container.container_name, False)) |
387 | |
388 | === modified file 'juju/providers/lxc/__init__.py' |
389 | --- juju/providers/lxc/__init__.py 2011-09-27 02:51:59 +0000 |
390 | +++ juju/providers/lxc/__init__.py 2011-09-29 06:40:31 +0000 |
391 | @@ -6,13 +6,13 @@ |
392 | from txzookeeper import ZookeeperClient |
393 | |
394 | from juju.errors import ProviderError, EnvironmentNotFound |
395 | -from juju.lib.lxc import LXCContainer |
396 | +from juju.lib.lxc import LXCContainer, get_containers |
397 | from juju.lib.zk import Zookeeper |
398 | from juju.providers.common.base import MachineProviderBase |
399 | from juju.providers.common.files import FileStorage |
400 | from juju.providers.common.connect import ZookeeperConnect |
401 | +from juju.providers.common.utils import get_user_authorized_keys |
402 | from juju.providers.lxc.agent import ManagedMachineAgent |
403 | -from juju.providers.lxc.container import get_containers |
404 | from juju.providers.lxc.machine import LocalMachine |
405 | from juju.providers.lxc.network import Network |
406 | from juju.providers.lxc.pkg import check_packages |
407 | @@ -107,6 +107,13 @@ |
408 | hierarchy = StateHierarchy(client, admin_identity, "local", "lxc") |
409 | yield hierarchy.initialize() |
410 | |
411 | + # Store user credentials from the running user |
412 | + try: |
413 | + public_key = get_user_authorized_keys(self.config) |
414 | + public_key = public_key.strip() |
415 | + except LookupError, e: |
416 | + raise ProviderError(str(e)) |
417 | + |
418 | # Startup the machine agent |
419 | log.info("Starting machine agent...") |
420 | pid_file = os.path.join(self._directory, "machine-agent.pid") |
421 | @@ -115,7 +122,9 @@ |
422 | zookeeper_hosts=zookeeper.address, |
423 | machine_id="0", |
424 | juju_directory=self._directory, |
425 | - log_file=log_file) |
426 | + log_file=log_file, |
427 | + juju_unit_namespace=self._qualified_name, |
428 | + public_key=public_key) |
429 | yield agent.start() |
430 | |
431 | log.info("Environment bootstrapped") |
432 | @@ -150,7 +159,7 @@ |
433 | def _destroy_containers(self): |
434 | container_map = yield get_containers(self._qualified_name) |
435 | for container_name in container_map: |
436 | - container = LXCContainer(container_name) |
437 | + container = LXCContainer(container_name, None, None, None) |
438 | if container_map[container.container_name]: |
439 | yield container.stop() |
440 | yield container.destroy() |
441 | |
442 | === modified file 'juju/providers/lxc/agent.py' |
443 | --- juju/providers/lxc/agent.py 2011-09-23 20:32:51 +0000 |
444 | +++ juju/providers/lxc/agent.py 2011-09-29 06:40:31 +0000 |
445 | @@ -1,5 +1,6 @@ |
446 | import errno |
447 | import os |
448 | +import pipes |
449 | import subprocess |
450 | import sys |
451 | |
452 | @@ -13,7 +14,9 @@ |
453 | |
454 | def __init__( |
455 | self, pid_file, zookeeper_hosts=None, machine_id="0", |
456 | - log_file=None, juju_directory="/var/lib/juju", juju_unit_namespace=""): |
457 | + log_file=None, juju_directory="/var/lib/juju", |
458 | + juju_unit_namespace="", |
459 | + public_key=None): |
460 | """ |
461 | :param pid_file: Path to file used to store process id. |
462 | :param machine_id: machine id for the machine agent (0 in lxc provider) |
463 | @@ -24,6 +27,8 @@ |
464 | a known a prefix to allow for multiple users and multiple |
465 | environments to create containers. The namespace should be |
466 | unique per user and per environment. |
467 | + :param public_key: An SSH public key (string) that will be |
468 | + used in the container for access. |
469 | """ |
470 | self._pid_file = pid_file |
471 | self._machine_id = machine_id |
472 | @@ -31,6 +36,7 @@ |
473 | self._juju_directory = juju_directory |
474 | self._juju_unit_namespace = juju_unit_namespace |
475 | self._log_file = log_file |
476 | + self._public_key = public_key |
477 | |
478 | @inlineCallbacks |
479 | def start(self): |
480 | @@ -52,6 +58,10 @@ |
481 | sys.executable, "-m", self.agent_module, |
482 | "-n", "--pidfile", self._pid_file, |
483 | "--logfile", self._log_file] |
484 | + |
485 | + if self._public_key: |
486 | + args.insert(1, "JUJU_PUBLIC_KEY=%s" % pipes.quote(self._public_key)) |
487 | + |
488 | yield deferToThread(subprocess.check_call, args) |
489 | |
490 | @inlineCallbacks |
491 | |
492 | === removed file 'juju/providers/lxc/container.py' |
493 | --- juju/providers/lxc/container.py 2011-09-23 17:25:37 +0000 |
494 | +++ juju/providers/lxc/container.py 1970-01-01 00:00:00 +0000 |
495 | @@ -1,31 +0,0 @@ |
496 | -from twisted.internet.threads import deferToThread |
497 | - |
498 | -import subprocess |
499 | - |
500 | - |
501 | -def get_containers(prefix=None): |
502 | - """Return a dictionary of containers key names to boolean values. |
503 | - |
504 | - A True value indicates the container is currently running. A False value |
505 | - indicates the container is defined but in the stopped state. |
506 | - |
507 | - :param prefix: Optionally specify a prefix that the container should |
508 | - match any returned containers. |
509 | - """ |
510 | - return deferToThread(_list_containers, prefix) |
511 | - |
512 | - |
513 | -def _list_containers(prefix): |
514 | - output = subprocess.check_output(["lxc-ls"]) |
515 | - containers = {} |
516 | - for i in filter(None, output.split("\n")): |
517 | - if i in containers: |
518 | - containers[i] = True |
519 | - else: |
520 | - containers[i] = False |
521 | - |
522 | - if prefix: |
523 | - remove = [k for k in containers.keys() if not k.startswith(prefix)] |
524 | - map(containers.pop, remove) |
525 | - |
526 | - return containers |
527 | |
528 | === modified file 'juju/providers/lxc/tests/test_container.py' |
529 | --- juju/providers/lxc/tests/test_container.py 2011-09-18 18:58:46 +0000 |
530 | +++ juju/providers/lxc/tests/test_container.py 2011-09-29 06:40:31 +0000 |
531 | @@ -3,8 +3,8 @@ |
532 | from twisted.internet.defer import inlineCallbacks |
533 | |
534 | from juju.lib.testing import TestCase |
535 | -from juju.providers.lxc.container import get_containers |
536 | - |
537 | +from juju.lib.lxc import get_containers |
538 | +from juju.lib import lxc |
539 | lxc_output_sample = "\ |
540 | calendarserver\ncaprica\ngemini\nreconnoiter\nvirgo\ncalendarserver\n" |
541 | |
542 | @@ -14,12 +14,12 @@ |
543 | @inlineCallbacks |
544 | def test_get_containers(self): |
545 | lxc_ls_mock = self.mocker.mock() |
546 | - self.patch(subprocess, "check_output", lxc_ls_mock) |
547 | + self.patch(lxc, "_cmd", lxc_ls_mock) |
548 | lxc_ls_mock(["lxc-ls"]) |
549 | - self.mocker.result(lxc_output_sample) |
550 | + self.mocker.result((0, lxc_output_sample)) |
551 | self.mocker.replay() |
552 | |
553 | - container_map = yield get_containers() |
554 | + container_map = yield get_containers(None) |
555 | self.assertEqual( |
556 | dict(caprica=False, gemini=False, reconnoiter=False, virgo=False, |
557 | calendarserver=True), |
558 | @@ -28,9 +28,9 @@ |
559 | @inlineCallbacks |
560 | def test_get_containers_with_prefix(self): |
561 | lxc_ls_mock = self.mocker.mock() |
562 | - self.patch(subprocess, "check_output", lxc_ls_mock) |
563 | + self.patch(lxc, "_cmd", lxc_ls_mock) |
564 | lxc_ls_mock(["lxc-ls"]) |
565 | - self.mocker.result(lxc_output_sample) |
566 | + self.mocker.result((0, lxc_output_sample)) |
567 | self.mocker.replay() |
568 | |
569 | container_map = yield get_containers("ca") |
570 | |
571 | === modified file 'juju/providers/lxc/tests/test_provider.py' |
572 | --- juju/providers/lxc/tests/test_provider.py 2011-09-27 02:51:59 +0000 |
573 | +++ juju/providers/lxc/tests/test_provider.py 2011-09-29 06:40:31 +0000 |
574 | @@ -11,6 +11,7 @@ |
575 | |
576 | from juju.errors import ProviderError, EnvironmentNotFound |
577 | from juju.lib.lxc import LXCContainer |
578 | +from juju.lib import lxc as lxc_lib |
579 | from juju.lib.testing import TestCase |
580 | from juju.tests.common import get_test_zookeeper_address |
581 | from juju.lib.zk import Zookeeper |
582 | @@ -43,8 +44,7 @@ |
583 | |
584 | def test_get_placement_policy(self): |
585 | """Lxc provider only supports local placement.""" |
586 | - self.assertEqual( |
587 | - self.provider.get_placement_policy(), "local") |
588 | + self.assertEqual(self.provider.get_placement_policy(), "local") |
589 | provider = MachineProvider( |
590 | "test", {"placement": "unassigned", |
591 | "data-dir": self.makeDir()}) |
592 | @@ -122,12 +122,12 @@ |
593 | user_name = pwd.getpwuid(os.getuid()).pw_name |
594 | # Mock container destruction, including stopping running contianers. |
595 | lxc_ls_mock = self.mocker.mock() |
596 | - self.patch(subprocess, "check_output", lxc_ls_mock) |
597 | + self.patch(lxc_lib, "_cmd", lxc_ls_mock) |
598 | lxc_ls_mock(["lxc-ls"]) |
599 | - self.mocker.result( |
600 | - "%(name)s-local-test-unit-1\n%(name)s-local-test-unit-2\n" |
601 | - "%(name)s-local-test-unit-3\n%(name)s-local-test-unit-1\n" |
602 | - "%(name)s-local-test-unit-2\n" % dict(name=user_name)) |
603 | + self.mocker.result((0, |
604 | + "%(name)s-local-test-unit-1\n%(name)s-local-test-unit-2\n" |
605 | + "%(name)s-local-test-unit-3\n%(name)s-local-test-unit-1\n" |
606 | + "%(name)s-local-test-unit-2\n" % dict(name=user_name))) |
607 | |
608 | mock_container = self.mocker.patch(LXCContainer) |
609 | mock_container.stop() |
610 | |
611 | === modified file 'misc/bash_completion.d/juju' |
612 | --- misc/bash_completion.d/juju 2011-09-15 18:58:54 +0000 |
613 | +++ misc/bash_completion.d/juju 2011-09-29 06:40:31 +0000 |
614 | @@ -6,7 +6,7 @@ |
615 | |
616 | COMPREPLY=() |
617 | cur=${COMP_WORDS[COMP_CWORD]} |
618 | - cmds='add-relation add-unit bootstrap debug-hooks debug-log deploy destroy-service expose open-tunnel remove-relation remove-unit resolved set shutdown ssh status terminate-machine unexpose upgrade-charm' |
619 | + cmds='add-relation add-unit bootstrap debug-hooks debug-log deploy destroy-environment destroy-service expose open-tunnel remove-relation remove-unit resolved set ssh status terminate-machine unexpose upgrade-charm' |
620 | globalOpts=( -h --verbose -v --log-file) |
621 | |
622 | # do ordinary expansion if we are anywhere after a -- argument |
623 | @@ -47,7 +47,7 @@ |
624 | optEnums=( ) |
625 | fixedWords=( ) |
626 | case "$cmd" in |
627 | - add-relation|debug-hooks|destory-service|expose-service|unexpose-service|open-tunnel|remove-relation|remove-unit|set|ssh|terminate-machine) |
628 | + add-relation|debug-hooks|destroy-environment|destory-service|expose-service|unexpose-service|open-tunnel|remove-relation|remove-unit|set|ssh|terminate-machine) |
629 | cmdOpts=( --environment ) |
630 | ;; |
631 | bootstrap) |
632 | |
633 | === modified file 'setup.py' |
634 | --- setup.py 2011-09-15 19:07:58 +0000 |
635 | +++ setup.py 2011-09-29 06:40:31 +0000 |
636 | @@ -30,6 +30,7 @@ |
637 | url="https://launchpad.net/juju", |
638 | license="GPL", |
639 | packages=find_packages(), |
640 | + package_data={"juju.lib.lxc": ["data/juju-create", "data/lxc.conf"]}, |
641 | scripts=glob("./bin/*"), |
642 | classifiers=[ |
643 | "Development Status :: 4 - Beta", |
What is the pre-req of this branch? I've already reviewed content in
this branch before.