Merge lp:~franciscosouza/pyjuju/juju-vpc into lp:pyjuju

Proposed by Francisco Souza
Status: Needs review
Proposed branch: lp:~franciscosouza/pyjuju/juju-vpc
Merge into: lp:pyjuju
Diff against target: 1999 lines (+752/-174)
46 files modified
juju/charm/repository.py (+3/-3)
juju/lib/http_client.py (+74/-0)
juju/machine/__init__.py (+2/-1)
juju/machine/tests/data/test_get_container (+1/-1)
juju/machine/tests/test_unit_deployment.py (+71/-1)
juju/machine/unit.py (+35/-15)
juju/providers/common/cloudinit.py (+35/-13)
juju/providers/common/launch.py (+10/-0)
juju/providers/common/tests/data/cloud_init_bootstrap (+2/-2)
juju/providers/common/tests/data/cloud_init_bootstrap_extra_env (+69/-0)
juju/providers/common/tests/data/cloud_init_bootstrap_zookeepers (+2/-2)
juju/providers/common/tests/data/cloud_init_branch (+1/-1)
juju/providers/common/tests/data/cloud_init_branch_trunk (+1/-1)
juju/providers/common/tests/data/cloud_init_distro (+1/-2)
juju/providers/common/tests/data/cloud_init_image (+35/-0)
juju/providers/common/tests/data/cloud_init_image_extra_env (+39/-0)
juju/providers/common/tests/data/cloud_init_no_machine_id (+2/-2)
juju/providers/common/tests/data/cloud_init_normal (+2/-2)
juju/providers/common/tests/data/cloud_init_ppa (+2/-2)
juju/providers/common/tests/data/cloud_init_ppa_apt_proxy (+2/-2)
juju/providers/common/tests/data/cloud_init_proposed (+1/-1)
juju/providers/common/tests/test_cloudinit.py (+36/-0)
juju/providers/ec2/__init__.py (+3/-2)
juju/providers/ec2/launch.py (+34/-18)
juju/providers/ec2/machine.py (+7/-3)
juju/providers/ec2/securitygroup.py (+36/-20)
juju/providers/ec2/tests/common.py (+13/-12)
juju/providers/ec2/tests/data/bootstrap_cloud_init (+2/-2)
juju/providers/ec2/tests/data/launch_cloud_init (+1/-1)
juju/providers/ec2/tests/data/launch_cloud_init_branch (+1/-1)
juju/providers/ec2/tests/data/launch_cloud_init_ip_address (+35/-0)
juju/providers/ec2/tests/data/launch_cloud_init_ppa (+1/-1)
juju/providers/ec2/tests/test_bootstrap.py (+3/-3)
juju/providers/ec2/tests/test_launch.py (+63/-26)
juju/providers/ec2/tests/test_machine.py (+18/-1)
juju/providers/ec2/tests/test_securitygroup.py (+39/-10)
juju/providers/ec2/tests/test_shutdown.py (+15/-10)
juju/providers/ec2/utils.py (+2/-2)
juju/providers/local/files.py (+2/-2)
juju/providers/orchestra/files.py (+2/-2)
juju/providers/orchestra/tests/data/bootstrap_user_data (+2/-2)
juju/providers/orchestra/tests/data/launch_user_data (+1/-1)
juju/unit/address.py (+13/-2)
juju/unit/charm.py (+2/-2)
juju/unit/deploy.py (+5/-0)
juju/unit/tests/test_address.py (+26/-0)
To merge this branch: bzr merge lp:~franciscosouza/pyjuju/juju-vpc
Reviewer Review Type Date Requested Status
Juju Engineering Pending
Review via email: mp+134169@code.launchpad.net

Description of the change

providers/ec2: support VPC

To use VPC, all the user need to do is add two new environment settings: vpc_id
and subnet_id.

This CL depends on three other CLs (related to txaws):

 - https://codereview.appspot.com/6826065/
 - https://codereview.appspot.com/6822097/
 - https://codereview.appspot.com/6814123/

** NOT FOR MERGE **

https://codereview.appspot.com/6850044/

To post a comment you must log in.
Revision history for this message
Chico (franciscossouza) wrote :

Reviewers: mp+134169_code.launchpad.net,

Message:
Please take a look.

Description:
providers/ec2: support VPC

To use VPC, all the user need to do is add two new environment settings:
vpc_id
and subnet_id.

This CL depends on three other CLs (related to txaws):

  - https://codereview.appspot.com/6826065/
  - https://codereview.appspot.com/6822097/
  - https://codereview.appspot.com/6814123/

https://code.launchpad.net/~franciscosouza/juju/juju-vpc/+merge/134169

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/6850044/

Affected files:
   A [revision details]
   M juju/providers/ec2/launch.py
   M juju/providers/ec2/securitygroup.py
   M juju/providers/ec2/tests/common.py
   M juju/providers/ec2/tests/test_bootstrap.py
   M juju/providers/ec2/tests/test_launch.py
   M juju/providers/ec2/tests/test_securitygroup.py
   M juju/providers/ec2/tests/test_shutdown.py

lp:~franciscosouza/pyjuju/juju-vpc updated
605. By Francisco Souza

provides/ec2: removed unused import

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
606. By Francisco Souza

providers/ec2: record machine private_ip_address (for VPC)

607. By Francisco Souza

machine: support private_ip_address

608. By Francisco Souza

providers/common: added private mode to cloudinit

In private mode, it uses private_ip_address for zookeeper.

609. By Francisco Souza

providers/common: use cloudinit's private mode

610. By Francisco Souza

providers/ec2: update launch with subnet to work in private mode

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
611. By Francisco Souza

providers/ec2: added private mode

Also adapt to new txaws api.

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
612. By Francisco Souza

providers/ec2: don't use group_name on describe_security_groups

I know that sucks, but we need a better solution later.

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
613. By Francisco Souza

cloudinit: added image source

This source should be used when juju comes pre-installed in the machine image.

Revision history for this message
Chico (franciscossouza) wrote :
Revision history for this message
Kapil Thangavelu (hazmat) wrote :

one comment regarding correctness.

https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py
File juju/providers/ec2/securitygroup.py (right):

https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py#newcode11
juju/providers/ec2/securitygroup.py:11: for group in d.result:
This is very suspect, deferred results should not be access directly,
this needs a callback registered or inlineCallbacks syntax.

https://codereview.appspot.com/6850044/

Revision history for this message
Chico (franciscossouza) wrote :

https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py
File juju/providers/ec2/securitygroup.py (right):

https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py#newcode11
juju/providers/ec2/securitygroup.py:11: for group in d.result:
On 2012/11/22 13:44:07, hazmat wrote:
> This is very suspect, deferred results should not be access directly,
this needs
> a callback registered or inlineCallbacks syntax.

Thanks, I will fix this.

I will also stop updating this CL using lbox, so you guys stop receiving
notifications. There is more than VPC in this CL now (proxy,
pre-installed juju, etc.), and we will probably keep using this fork. I
hope that we will be able to send smaller patchs for merging in the
(near) future.

https://codereview.appspot.com/6850044/

Revision history for this message
Kapil Thangavelu (hazmat) wrote :

no worries about using lbox, i don't mind having a look on occasion, i
understood from the merge proposal that it wasn't intended for merge. i've
been going through the txaws branches as well, which look good but i'll
need to rope in one more reviewer for them i think. i'd like to get some
additional functionality there for txaws to be able to create vpc, with a
goal that juju can create the vpc itself. cheers.

On Thu, Nov 22, 2012 at 8:58 AM, <email address hidden> wrote:

>
> https://codereview.appspot.**com/6850044/diff/7017/juju/**
> providers/ec2/securitygroup.py<https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py>
> File juju/providers/ec2/**securitygroup.py (right):
>
> https://codereview.appspot.**com/6850044/diff/7017/juju/**
> providers/ec2/securitygroup.**py#newcode11<https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py#newcode11>
> juju/providers/ec2/**securitygroup.py:11: for group in d.result:
> On 2012/11/22 13:44:07, hazmat wrote:
>
>> This is very suspect, deferred results should not be access directly,
>>
> this needs
>
>> a callback registered or inlineCallbacks syntax.
>>
>
> Thanks, I will fix this.
>
> I will also stop updating this CL using lbox, so you guys stop receiving
> notifications. There is more than VPC in this CL now (proxy,
> pre-installed juju, etc.), and we will probably keep using this fork. I
> hope that we will be able to send smaller patchs for merging in the
> (near) future.
>
> https://codereview.appspot.**com/6850044/<https://codereview.appspot.com/6850044/>
>

Revision history for this message
Chico (franciscossouza) wrote :

On 2012/11/22 14:26:33, hazmat wrote:
> no worries about using lbox, i don't mind having a look on occasion, i
> understood from the merge proposal that it wasn't intended for merge.
i've
> been going through the txaws branches as well, which look good but
i'll
> need to rope in one more reviewer for them i think. i'd like to get
some
> additional functionality there for txaws to be able to create vpc,
with a
> goal that juju can create the vpc itself. cheers.

Indeed. txaws branches are there for merging, I think they're quite
there. Are you able to review and merge them? I'd like to move forward
in getting those branches merged.

Regards,
Francisco

https://codereview.appspot.com/6850044/

lp:~franciscosouza/pyjuju/juju-vpc updated
614. By Francisco Souza

providers/ec2: don't use deferred.result

Revision history for this message
Chico (franciscossouza) wrote :

Please take a look.

https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py
File juju/providers/ec2/securitygroup.py (right):

https://codereview.appspot.com/6850044/diff/7017/juju/providers/ec2/securitygroup.py#newcode11
juju/providers/ec2/securitygroup.py:11: for group in d.result:
On 2012/11/22 13:44:07, hazmat wrote:
> This is very suspect, deferred results should not be access directly,
this needs
> a callback registered or inlineCallbacks syntax.

Done.

https://codereview.appspot.com/6850044/

lp:~franciscosouza/pyjuju/juju-vpc updated
615. By Francisco Souza

providers/common: added extra environment support for cloud init

616. By Francisco Souza

machine: added support for extra environment variables in unit deployment

617. By Francisco Souza

providers/common: launching vms with http and https proxy in upstart file

618. By Francisco Souza

unit/deploy: read http_proxy and https_proxy from os environment

These values are used to add new units, setting http_proxy and https_proxy in
upstart config file.

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
619. By Francisco Souza

providers/common: set extra environment on provision agent too

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
620. By Francisco Souza

lib: added http_client

This will wrap getPage and downloadPage from twisted, enabling proxy.

621. By Francisco Souza

all: use download_page and get_page from lib.http_client

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
622. By Francisco Souza

lib/http_client: don't use proxy to connect to localhost, 127.0.0.1 and 169.254.169.254

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
623. By Francisco Souza

lib/http_client: fix stupid error

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
624. By Francisco Souza

providers, unit: added private mode for address

A VPC instance does not have public DNS name, so the metadata url for this data
returns 404.

Revision history for this message
Chico (franciscossouza) wrote :
lp:~franciscosouza/pyjuju/juju-vpc updated
625. By Francisco Souza

unit/deploy: set JUJU_PRIVATE on service upstart file

Revision history for this message
Chico (franciscossouza) wrote :

Unmerged revisions

625. By Francisco Souza

unit/deploy: set JUJU_PRIVATE on service upstart file

624. By Francisco Souza

providers, unit: added private mode for address

A VPC instance does not have public DNS name, so the metadata url for this data
returns 404.

623. By Francisco Souza

lib/http_client: fix stupid error

622. By Francisco Souza

lib/http_client: don't use proxy to connect to localhost, 127.0.0.1 and 169.254.169.254

621. By Francisco Souza

all: use download_page and get_page from lib.http_client

620. By Francisco Souza

lib: added http_client

This will wrap getPage and downloadPage from twisted, enabling proxy.

619. By Francisco Souza

providers/common: set extra environment on provision agent too

618. By Francisco Souza

unit/deploy: read http_proxy and https_proxy from os environment

These values are used to add new units, setting http_proxy and https_proxy in
upstart config file.

617. By Francisco Souza

providers/common: launching vms with http and https proxy in upstart file

616. By Francisco Souza

machine: added support for extra environment variables in unit deployment

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'juju/charm/repository.py'
2--- juju/charm/repository.py 2012-08-23 23:57:09 +0000
3+++ juju/charm/repository.py 2012-11-23 21:00:27 +0000
4@@ -7,7 +7,6 @@
5 import yaml
6
7 from twisted.internet.defer import fail, inlineCallbacks, returnValue, succeed
8-from twisted.web.client import downloadPage, getPage
9 from twisted.web.error import Error
10 from txaws.client.ssl import VerifyingContextFactory
11
12@@ -15,6 +14,7 @@
13 from juju.charm.url import CharmURL
14 from juju.errors import FileNotFound
15 from juju.lib import under
16+from juju.lib.http_client import download_page, get_page
17
18 from .errors import (
19 CharmNotFound, CharmError, RepositoryNotFound, ServiceConfigValueError)
20@@ -129,7 +129,7 @@
21 self.url_base, urllib.quote(charm_id))
22 try:
23 host = urlparse.urlparse(url).hostname
24- all_info = json.loads((yield getPage(url, contextFactory=VerifyingContextFactory(host))))
25+ all_info = json.loads((yield get_page(url, contextFactory=VerifyingContextFactory(host))))
26 charm_info = all_info[charm_id]
27 for warning in charm_info.get("warnings", []):
28 log.warning("%s: %s", charm_id, warning)
29@@ -152,7 +152,7 @@
30 downloading_path = f.name
31 host = urlparse.urlparse(url).hostname
32 try:
33- yield downloadPage(url, downloading_path, contextFactory=VerifyingContextFactory(host))
34+ yield download_page(url, downloading_path, contextFactory=VerifyingContextFactory(host))
35 except Error:
36 raise CharmNotFound(self.url_base, charm_url)
37 os.rename(downloading_path, cache_path)
38
39=== added file 'juju/lib/http_client.py'
40--- juju/lib/http_client.py 1970-01-01 00:00:00 +0000
41+++ juju/lib/http_client.py 2012-11-23 21:00:27 +0000
42@@ -0,0 +1,74 @@
43+# -*- coding: utf-8 -*-
44+import os
45+import urlparse
46+
47+from twisted.internet import reactor
48+from twisted.internet.defer import Deferred
49+from twisted.internet.endpoints import TCP4ClientEndpoint
50+from twisted.internet.protocol import connectionDone, Protocol
51+from twisted.web.client import downloadPage, getPage, ProxyAgent, ResponseDone
52+from twisted.web.http import PotentialDataLoss
53+
54+
55+class Receiver(Protocol):
56+ finished = None
57+ _filename = None
58+
59+ def __init__(self, filename=None):
60+ self.received = ""
61+ if filename:
62+ self._filename = filename
63+
64+ def dataReceived(self, data):
65+ self.received += data
66+
67+ def connectionLost(self, reason=connectionDone):
68+ reason.trap(ResponseDone, PotentialDataLoss, connectionDone)
69+ d = self.finished
70+ self.finished = None
71+ if self._filename:
72+ file = open(self._filename, "w")
73+ try:
74+ file.write(self.received)
75+ finally:
76+ file.close()
77+ d.callback(self.received)
78+
79+
80+def response_saver(path):
81+ def save_response(response):
82+ receiver = Receiver(path)
83+ receiver.finished = d = Deferred()
84+ response.deliverBody(receiver)
85+ return d
86+ return save_response
87+
88+
89+def _deferred_get(url, proxy_endpoint):
90+ proxy_url = urlparse.urlparse(proxy_endpoint)
91+ endpoint = TCP4ClientEndpoint(reactor, proxy_url.hostname, proxy_url.port)
92+ agent = ProxyAgent(endpoint)
93+ return agent.request("GET", url)
94+
95+
96+def download_page(url, path, *args, **kwargs):
97+ host = urlparse.urlparse(url).hostname
98+ proxy_var = url.startswith("https://") and "https_proxy" or "http_proxy"
99+ proxy_endpoint = os.environ.get(proxy_var)
100+ if proxy_endpoint and host not in ("localhost", "127.0.0.1", "169.254.169.254"):
101+ d = _deferred_get(url, proxy_endpoint)
102+ d.addCallback(response_saver(path))
103+ return d
104+ return downloadPage(url, path, *args, **kwargs)
105+
106+
107+def get_page(url, *args, **kwargs):
108+ host = urlparse.urlparse(url).hostname
109+ proxy_var = url.startswith("https://") and "https_proxy" or "http_proxy"
110+ proxy_endpoint = os.environ.get(proxy_var)
111+ if proxy_endpoint and host not in ("localhost", "127.0.0.1", "169.254.169.254"):
112+ d = _deferred_get(url, proxy_endpoint)
113+ d.addCallback(response_saver(None))
114+ return d
115+ else:
116+ return getPage(url, *args, **kwargs)
117
118=== modified file 'juju/machine/__init__.py'
119--- juju/machine/__init__.py 2012-01-31 19:57:53 +0000
120+++ juju/machine/__init__.py 2012-11-23 21:00:27 +0000
121@@ -9,9 +9,10 @@
122 """
123
124 def __init__(self, instance_id, dns_name=None, private_dns_name=None,
125- state="unknown"):
126+ state="unknown", private_ip_address=None):
127 self.instance_id = instance_id
128 # ideally this would be ip_address, but txaws doesn't expose it.
129 self.dns_name = dns_name
130 self.private_dns_name = private_dns_name
131+ self.private_ip_address = private_ip_address
132 self.state = state
133
134=== modified file 'juju/machine/tests/data/test_get_container'
135--- juju/machine/tests/data/test_get_container 2012-10-09 18:55:55 +0000
136+++ juju/machine/tests/data/test_get_container 2012-11-23 21:00:27 +0000
137@@ -7,7 +7,7 @@
138 machine-data: {juju-provider-type: null, juju-zookeeper-hosts: 'localhost:2181',
139 machine-id: null}
140 output: {all: '| tee -a /var/log/cloud-init-output.log'}
141-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper]
142+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, python-txaws]
143 runcmd: [sudo apt-get install -y python-txzookeeper, sudo mkdir -p /usr/lib/juju,
144 'cd /usr/lib/juju && sudo /usr/bin/bzr co --lightweight lp:juju
145 juju', cd /usr/lib/juju/juju && sudo python setup.py develop, sudo mkdir -p /var/lib/juju,
146
147=== modified file 'juju/machine/tests/test_unit_deployment.py'
148--- juju/machine/tests/test_unit_deployment.py 2012-10-05 13:55:09 +0000
149+++ juju/machine/tests/test_unit_deployment.py 2012-11-23 21:00:27 +0000
150@@ -15,7 +15,7 @@
151 from juju.lib.lxc.tests.test_lxc import uses_sudo
152 from juju.lib.mocker import ANY, KWARGS, MATCH
153 from juju.lib.upstart import UpstartService
154-from juju.machine.unit import UnitMachineDeployment, UnitContainerDeployment
155+from juju.machine.unit import ExtraEnvironmentDeployment, UnitMachineDeployment, UnitContainerDeployment
156 from juju.machine.errors import UnitDeploymentError
157 from juju.machine import tests
158 from juju.tests.common import get_test_zookeeper_address
159@@ -143,6 +143,54 @@
160 d.addCallback(verify_upstart)
161 return d
162
163+ def test_service_unit_start_with_extra_environment(self):
164+ """
165+ Starting a service unit will result in a unit workspace being created
166+ if it does not exist and a running service unit agent.
167+ """
168+ self.setup_mock()
169+ self.mock_install()
170+ self.mock_is_running(False)
171+ self.mock_start()
172+ self.mocker.replay()
173+
174+ self.deployment.set_env("http_proxy", "http://myproxy.com:3128/")
175+ d = self.deployment.start(
176+ "123", get_test_zookeeper_address(), self.bundle)
177+
178+ def verify_upstart(_):
179+ conf_path = os.path.join(self.init_dir, "juju-wordpress-0.conf")
180+ with open(conf_path) as f:
181+ lines = f.readlines()
182+
183+ env = []
184+ for line in lines:
185+ if line.startswith("env "):
186+ env.append(line[4:-1].split("=", 1))
187+ if line.startswith("exec "):
188+ exec_ = line[5:-1]
189+
190+ env = dict((k, v.strip('"')) for (k, v) in env)
191+ env.pop("PYTHONPATH")
192+ self.assertEquals(env, {
193+ "JUJU_HOME": self.juju_directory,
194+ "JUJU_UNIT_NAME": self.unit_name,
195+ "JUJU_ZOOKEEPER": get_test_zookeeper_address(),
196+ "JUJU_MACHINE_ID": "123",
197+ "http_proxy": "http://myproxy.com:3128/"})
198+
199+ log_file = os.path.join(
200+ self.deployment.directory, "charm.log")
201+ command = " ".join([
202+ "/usr/bin/python", "-m", "juju.agents.dummy", "--nodaemon",
203+ "--logfile", log_file, "--session-file",
204+ "/var/run/juju/unit-wordpress-0-agent.zksession",
205+ ">> /tmp/juju-wordpress-0.output 2>&1"])
206+ self.assertEquals(exec_, command)
207+ d.addCallback(verify_upstart)
208+ self.deployment.unset_env("http_proxy")
209+ return d
210+
211 @inlineCallbacks
212 def test_service_unit_destroy(self):
213 """
214@@ -426,3 +474,25 @@
215
216 container, rootfs = yield self.unit_deploy._get_container(
217 "0", cloud_init)
218+
219+
220+class ExtraEnvironmentTest(RepositoryTestBase):
221+
222+ def test_set_env(self):
223+ deploy = ExtraEnvironmentDeployment()
224+ deploy.set_env("http_proxy", "http://myproxy.com:3128/")
225+ self.assertEquals(deploy._extra_environment["http_proxy"], "http://myproxy.com:3128/")
226+
227+ def test_unset_env(self):
228+ deploy = ExtraEnvironmentDeployment()
229+ deploy.set_env("http_proxy", "http://myproxy.com:3128/")
230+ deploy.unset_env("http_proxy")
231+ self.assertNotIn("http_proxy", deploy._extra_environment)
232+
233+ def test_machine_deployment_extends_extra_environment(self):
234+ assert issubclass(UnitMachineDeployment, ExtraEnvironmentDeployment), \
235+ "UnitMachineDeployment should inherit from ExtraEnvironmentDeployment!"
236+
237+ def test_container_deployment_extends_extra_environment(self):
238+ assert issubclass(UnitContainerDeployment, ExtraEnvironmentDeployment), \
239+ "UnitContainerDeployment should inherit from ExtraEnvironmentDeployment!"
240
241=== modified file 'juju/machine/unit.py'
242--- juju/machine/unit.py 2012-09-27 00:42:32 +0000
243+++ juju/machine/unit.py 2012-11-23 21:00:27 +0000
244@@ -8,7 +8,7 @@
245
246 from juju.charm.bundle import CharmBundle
247 from juju.errors import ServiceError
248-from juju.lib.lxc import LXCContainer, get_containers, LXCError
249+from juju.lib.lxc import LXCContainer, get_containers
250 from juju.lib.twistutils import get_module_directory
251 from juju.lib.upstart import UpstartService
252 from juju.providers.common.cloudinit import CloudInit
253@@ -27,20 +27,33 @@
254 return UnitMachineDeployment
255
256
257-def _get_environment(unit_name, juju_home, machine_id, zookeeper_hosts):
258- environ = dict()
259- environ["JUJU_MACHINE_ID"] = str(machine_id)
260- environ["JUJU_UNIT_NAME"] = unit_name
261- environ["JUJU_HOME"] = juju_home
262- environ["JUJU_ZOOKEEPER"] = zookeeper_hosts
263- environ["PYTHONPATH"] = ":".join(
264+def _get_environment(unit_name, juju_home, machine_id, zookeeper_hosts, base=None):
265+ if not base:
266+ base = {}
267+ base["JUJU_MACHINE_ID"] = str(machine_id)
268+ base["JUJU_UNIT_NAME"] = unit_name
269+ base["JUJU_HOME"] = juju_home
270+ base["JUJU_ZOOKEEPER"] = zookeeper_hosts
271+ base["PYTHONPATH"] = ":".join(
272 filter(None, [
273 os.path.dirname(get_module_directory(juju)),
274 os.environ.get("PYTHONPATH")]))
275- return environ
276-
277-
278-class UnitMachineDeployment(object):
279+ return base
280+
281+
282+class ExtraEnvironmentDeployment(object):
283+
284+ def __init__(self):
285+ self._extra_environment = {}
286+
287+ def set_env(self, name, value):
288+ self._extra_environment[name] = value
289+
290+ def unset_env(self, name):
291+ del self._extra_environment[name]
292+
293+
294+class UnitMachineDeployment(ExtraEnvironmentDeployment):
295 """ Deploy a unit directly onto a machine.
296
297 A service unit deployed directly to a machine has full access
298@@ -54,6 +67,7 @@
299 unit_agent_module = "juju.agents.unit"
300
301 def __init__(self, unit_name, juju_home):
302+ super(UnitMachineDeployment, self).__init__()
303 assert ".." not in unit_name, "Invalid Unit Name"
304 self.unit_name = unit_name
305 self.juju_home = juju_home
306@@ -72,7 +86,7 @@
307 self.service.set_description(
308 "Juju unit agent for %s" % self.unit_name)
309 self.service.set_environ(_get_environment(
310- self.unit_name, self.juju_home, machine_id, zookeeper_hosts))
311+ self.unit_name, self.juju_home, machine_id, zookeeper_hosts, self._extra_environment))
312 self.service.set_command(" ".join((
313 "/usr/bin/python", "-m", self.unit_agent_module,
314 "--nodaemon",
315@@ -130,7 +144,7 @@
316 "juju-%s" % self.unit_path_name, use_sudo=True)
317
318
319-class UnitContainerDeployment(object):
320+class UnitContainerDeployment(ExtraEnvironmentDeployment):
321 """Deploy a service unit in a container.
322
323 Units deployed in a container have strong isolation between
324@@ -145,6 +159,7 @@
325 """
326
327 def __init__(self, unit_name, juju_home):
328+ super(UnitContainerDeployment, self).__init__()
329 self.unit_name = unit_name
330 self.juju_home = juju_home
331 self.unit_path_name = unit_name.replace("/", "-")
332@@ -189,7 +204,7 @@
333 (zk, port) = zk.split(':')
334 else:
335 port = 2181
336- zks.append((zk,port))
337+ zks.append((zk, port))
338
339 cloud_init.set_zookeeper_hosts(zks)
340 # XXX Very hard to access the provider's notion of network
341@@ -197,6 +212,9 @@
342 # the apt-cacher-ng since this is meant for local provider.
343 cloud_init.set_apt_proxy('http://%s:3142' % zks[0][0])
344
345+ for name, value in self._extra_environment.iteritems():
346+ cloud_init.set_env(name, value)
347+
348 if self._juju_origin:
349 if self._juju_origin.startswith("lp:"):
350 cloud_init.set_juju_source(branch=self._juju_origin)
351@@ -204,6 +222,8 @@
352 cloud_init.set_juju_source(ppa=True)
353 elif self._juju_origin == "proposed":
354 cloud_init.set_juju_source(proposed=True)
355+ elif self._juju_origin == "image":
356+ cloud_init.set_juju_source(image=True)
357 else:
358 # Ignore other values, just use the distro for sanity
359 cloud_init.set_juju_source(distro=True)
360
361=== modified file 'juju/providers/common/cloudinit.py'
362--- juju/providers/common/cloudinit.py 2012-10-09 18:55:55 +0000
363+++ juju/providers/common/cloudinit.py 2012-11-23 21:00:27 +0000
364@@ -13,6 +13,7 @@
365 PPA = "ppa"
366 BRANCH = "branch"
367 PROPOSED = "proposed"
368+IMAGE = "image"
369
370
371 def _branch_install_scripts(branch):
372@@ -45,11 +46,13 @@
373 b64encode(serializer.dump(constraints.data)), provider_type)]
374
375
376-def _machine_scripts(machine_id, zookeeper_hosts):
377+def _machine_scripts(machine_id, zookeeper_hosts, extra_env=None):
378 service = UpstartService("juju-machine-agent")
379 service.set_description("Juju machine agent")
380- service.set_environ(
381- {"JUJU_MACHINE_ID": machine_id, "JUJU_ZOOKEEPER": zookeeper_hosts})
382+ env = {"JUJU_MACHINE_ID": machine_id, "JUJU_ZOOKEEPER": zookeeper_hosts}
383+ if extra_env:
384+ env.update(extra_env)
385+ service.set_environ(env)
386 service.set_command(
387 "python -m juju.agents.machine --nodaemon "
388 "--logfile /var/log/juju/machine-agent.log "
389@@ -78,10 +81,13 @@
390 "/var/run/juju/unit-%s-agent.zksession" % unit_path_name]))
391 return service.get_cloud_init_commands()
392
393-def _provision_scripts(zookeeper_hosts):
394+def _provision_scripts(zookeeper_hosts, extra_env=None):
395 service = UpstartService("juju-provision-agent")
396 service.set_description("Juju provisioning agent")
397- service.set_environ({"JUJU_ZOOKEEPER": zookeeper_hosts})
398+ env = {"JUJU_ZOOKEEPER": zookeeper_hosts}
399+ if extra_env:
400+ env.update(extra_env)
401+ service.set_environ(env)
402 service.set_command(
403 "python -m juju.agents.provision --nodaemon "
404 "--logfile /var/log/juju/provision-agent.log "
405@@ -175,6 +181,8 @@
406 self._unit_name = None
407 self._juju_home = None
408 self._apt_proxy = None
409+ self._extra_environment = {}
410+ self.private = False
411
412 def add_ssh_key(self, key):
413 """Add an SSH public key.
414@@ -190,8 +198,14 @@
415 self._zookeeper = True
416 self._provision = True
417
418+ def set_env(self, name, value):
419+ self._extra_environment[name] = value
420+
421+ def unset_env(self, name):
422+ del self._extra_environment[name]
423+
424 def set_juju_source(
425- self, branch=None, ppa=False, distro=False, proposed=False):
426+ self, branch=None, ppa=False, distro=False, proposed=False, image=False):
427 """Set the version of juju the machine should run.
428
429 :param branch: location from which to check out juju; for example,
430@@ -213,9 +227,12 @@
431 Note that you don't need to call this method; the juju source
432 defaults to what is returned by `get_default_origin`.
433 """
434- if len(filter(None, (branch, ppa, distro, proposed))) != 1:
435+ if len(filter(None, (branch, ppa, distro, proposed, image))) != 1:
436 raise CloudInitError("Please specify one source")
437- if branch:
438+ if image:
439+ self._origin = IMAGE
440+ self._origin_url = None
441+ elif branch:
442 self._origin = BRANCH
443 self._origin_url = branch
444 elif ppa:
445@@ -282,7 +299,10 @@
446 You don't have to set this, so long as the machine you are starting is
447 itself a zookeeper instance.
448 """
449- self._zookeeper_hosts = [m.private_dns_name for m in machines]
450+ if self.private:
451+ self._zookeeper_hosts = [m.private_ip_address for m in machines]
452+ else:
453+ self._zookeeper_hosts = [m.private_dns_name for m in machines]
454
455 def set_zookeeper_hosts(self, hosts):
456 """Specify master :class:`juju.machine.ProviderMachine` instances.
457@@ -367,18 +387,20 @@
458 def _collect_packages(self):
459 packages = [
460 "bzr", "byobu", "tmux", "python-setuptools", "python-twisted",
461- "python-txaws", "python-zookeeper"]
462+ "python-zookeeper"]
463 if self._zookeeper:
464 packages.extend([
465 "default-jre-headless", "zookeeper", "zookeeperd"])
466 if self._origin in (DISTRO, PPA, PROPOSED):
467 packages.append("juju")
468+ if self._origin != IMAGE:
469+ packages.append("python-txaws")
470 return packages
471
472 def _collect_repositories(self):
473 if self._origin == PROPOSED:
474 return ["deb $MIRROR $RELEASE-proposed main universe"]
475- if self._origin != DISTRO:
476+ if self._origin not in (DISTRO, IMAGE):
477 return ["ppa:juju/pkgs"]
478 return []
479
480@@ -392,7 +414,7 @@
481 self._provider_type))
482 if self._machine_id:
483 scripts.extend(_machine_scripts(
484- self._machine_id, self._join_zookeeper_hosts()))
485+ self._machine_id, self._join_zookeeper_hosts(), self._extra_environment))
486 if self._unit_name:
487 scripts.extend(_unit_scripts(
488 self._machine_id,
489@@ -400,7 +422,7 @@
490 self._join_zookeeper_hosts(),
491 self._juju_home))
492 if self._provision:
493- scripts.extend(_provision_scripts(self._join_zookeeper_hosts()))
494+ scripts.extend(_provision_scripts(self._join_zookeeper_hosts(), self._extra_environment))
495 return scripts
496
497 def _collect_machine_data(self):
498
499=== modified file 'juju/providers/common/launch.py'
500--- juju/providers/common/launch.py 2012-07-18 20:04:16 +0000
501+++ juju/providers/common/launch.py 2012-11-23 21:00:27 +0000
502@@ -92,6 +92,7 @@
503 """
504 config = self._provider.config
505 cloud_init = CloudInit()
506+ cloud_init.private = config.get("vpc_id") is not None
507 cloud_init.add_ssh_key(get_user_authorized_keys(config))
508 cloud_init.set_machine_id(machine_id)
509 cloud_init.set_zookeeper_machines(zookeepers)
510@@ -103,9 +104,18 @@
511 cloud_init.set_juju_source(ppa=True)
512 elif origin == "proposed":
513 cloud_init.set_juju_source(proposed=True)
514+ elif origin == "image":
515+ cloud_init.set_juju_source(image=True)
516 else:
517 # Ignore other values, just use the distro for sanity
518 cloud_init.set_juju_source(distro=True)
519+ proxy = config.get("proxy")
520+ if proxy:
521+ cloud_init.set_env("http_proxy", proxy)
522+ cloud_init.set_env("https_proxy", proxy)
523+ private = config.get("vpc_id") is not None
524+ if private:
525+ cloud_init.set_env("JUJU_PRIVATE", "private")
526 if self._master:
527 cloud_init.enable_bootstrap()
528 cloud_init.set_zookeeper_secret(config["admin-secret"])
529
530=== modified file 'juju/providers/common/tests/data/cloud_init_bootstrap'
531--- juju/providers/common/tests/data/cloud_init_bootstrap 2012-08-23 16:14:42 +0000
532+++ juju/providers/common/tests/data/cloud_init_bootstrap 2012-11-23 21:00:27 +0000
533@@ -4,8 +4,8 @@
534 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'localhost:2181',
535 machine-id: passport}
536 output: {all: '| tee -a /var/log/cloud-init-output.log'}
537-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
538- default-jre-headless, zookeeper, zookeeperd, juju]
539+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
540+ default-jre-headless, zookeeper, zookeeperd, juju, python-txaws]
541 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'juju-admin initialize
542 --instance-id=token --admin-identity=admin:19vlzY4Vc3q4Ew5OsCwKYqrq1HI= --constraints-data=e2NwdTogJzIwJywgcHJvdmlkZXItdHlwZTogZHVtbXksIHVidW50dS1zZXJpZXM6IGFzdG9uaXNoaW5nfQo=
543 --provider-type=dummy', 'cat >> /etc/init/juju-machine-agent.conf <<EOF
544
545=== added file 'juju/providers/common/tests/data/cloud_init_bootstrap_extra_env'
546--- juju/providers/common/tests/data/cloud_init_bootstrap_extra_env 1970-01-01 00:00:00 +0000
547+++ juju/providers/common/tests/data/cloud_init_bootstrap_extra_env 2012-11-23 21:00:27 +0000
548@@ -0,0 +1,69 @@
549+#cloud-config
550+apt_update: true
551+apt_upgrade: true
552+machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'localhost:2181',
553+ machine-id: passport}
554+output: {all: '| tee -a /var/log/cloud-init-output.log'}
555+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
556+ default-jre-headless, zookeeper, zookeeperd, juju, python-txaws]
557+runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'juju-admin initialize
558+ --instance-id=token --admin-identity=admin:19vlzY4Vc3q4Ew5OsCwKYqrq1HI= --constraints-data=e2NwdTogJzIwJywgcHJvdmlkZXItdHlwZTogZHVtbXksIHVidW50dS1zZXJpZXM6IGFzdG9uaXNoaW5nfQo=
559+ --provider-type=dummy', 'cat >> /etc/init/juju-machine-agent.conf <<EOF
560+
561+ description "Juju machine agent"
562+
563+ author "Juju Team <juju@lists.ubuntu.com>"
564+
565+
566+ start on runlevel [2345]
567+
568+ stop on runlevel [!2345]
569+
570+ respawn
571+
572+
573+ env JUJU_MACHINE_ID="passport"
574+
575+ env JUJU_ZOOKEEPER="localhost:2181"
576+
577+ env http_proxy="http://myproxy.com:3128/"
578+
579+ env https_proxy="http://myproxy.com:3128/"
580+
581+
582+ exec python -m juju.agents.machine --nodaemon --logfile /var/log/juju/machine-agent.log
583+ --session-file /var/run/juju/machine-agent.zksession >> /tmp/juju-machine-agent.output
584+ 2>&1
585+
586+ EOF
587+
588+ ', /sbin/start juju-machine-agent, 'cat >> /etc/init/juju-provision-agent.conf
589+ <<EOF
590+
591+ description "Juju provisioning agent"
592+
593+ author "Juju Team <juju@lists.ubuntu.com>"
594+
595+
596+ start on runlevel [2345]
597+
598+ stop on runlevel [!2345]
599+
600+ respawn
601+
602+
603+ env JUJU_ZOOKEEPER="localhost:2181"
604+
605+ env http_proxy="http://myproxy.com:3128/"
606+
607+ env https_proxy="http://myproxy.com:3128/"
608+
609+
610+ exec python -m juju.agents.provision --nodaemon --logfile /var/log/juju/provision-agent.log
611+ --session-file /var/run/juju/provision-agent.zksession >> /tmp/juju-provision-agent.output
612+ 2>&1
613+
614+ EOF
615+
616+ ', /sbin/start juju-provision-agent]
617+ssh_authorized_keys: [chubb]
618
619=== modified file 'juju/providers/common/tests/data/cloud_init_bootstrap_zookeepers'
620--- juju/providers/common/tests/data/cloud_init_bootstrap_zookeepers 2012-08-23 16:14:42 +0000
621+++ juju/providers/common/tests/data/cloud_init_bootstrap_zookeepers 2012-11-23 21:00:27 +0000
622@@ -4,8 +4,8 @@
623 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181,localhost:2181',
624 machine-id: passport}
625 output: {all: '| tee -a /var/log/cloud-init-output.log'}
626-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
627- default-jre-headless, zookeeper, zookeeperd, juju]
628+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
629+ default-jre-headless, zookeeper, zookeeperd, juju, python-txaws]
630 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'juju-admin initialize
631 --instance-id=token --admin-identity=admin:19vlzY4Vc3q4Ew5OsCwKYqrq1HI= --constraints-data=e2NwdTogJzIwJywgcHJvdmlkZXItdHlwZTogZHVtbXksIHVidW50dS1zZXJpZXM6IGFzdG9uaXNoaW5nfQo=
632 --provider-type=dummy', 'cat >> /etc/init/juju-machine-agent.conf <<EOF
633
634=== modified file 'juju/providers/common/tests/data/cloud_init_branch'
635--- juju/providers/common/tests/data/cloud_init_branch 2012-10-09 18:55:55 +0000
636+++ juju/providers/common/tests/data/cloud_init_branch 2012-11-23 21:00:27 +0000
637@@ -6,7 +6,7 @@
638 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
639 machine-id: passport}
640 output: {all: '| tee -a /var/log/cloud-init-output.log'}
641-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper]
642+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, python-txaws]
643 runcmd: [sudo apt-get install -y python-txzookeeper, sudo mkdir -p /usr/lib/juju,
644 'cd /usr/lib/juju && sudo /usr/bin/bzr co --lightweight lp:blah/juju/blah-blah juju', cd /usr/lib/juju/juju
645 && sudo python setup.py develop, sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju,
646
647=== modified file 'juju/providers/common/tests/data/cloud_init_branch_trunk'
648--- juju/providers/common/tests/data/cloud_init_branch_trunk 2012-10-09 18:55:55 +0000
649+++ juju/providers/common/tests/data/cloud_init_branch_trunk 2012-11-23 21:00:27 +0000
650@@ -6,7 +6,7 @@
651 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
652 machine-id: passport}
653 output: {all: '| tee -a /var/log/cloud-init-output.log'}
654-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper]
655+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, python-txaws]
656 runcmd: [sudo apt-get install -y python-txzookeeper, sudo mkdir -p /usr/lib/juju,
657 'cd /usr/lib/juju && sudo /usr/bin/bzr co --lightweight lp:juju juju', cd /usr/lib/juju/juju &&
658 sudo python setup.py develop, sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju,
659
660=== modified file 'juju/providers/common/tests/data/cloud_init_distro'
661--- juju/providers/common/tests/data/cloud_init_distro 2012-08-23 16:14:42 +0000
662+++ juju/providers/common/tests/data/cloud_init_distro 2012-11-23 21:00:27 +0000
663@@ -4,8 +4,7 @@
664 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
665 machine-id: passport}
666 output: {all: '| tee -a /var/log/cloud-init-output.log'}
667-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
668- juju]
669+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, juju, python-txaws]
670 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf
671 <<EOF
672
673
674=== added file 'juju/providers/common/tests/data/cloud_init_image'
675--- juju/providers/common/tests/data/cloud_init_image 1970-01-01 00:00:00 +0000
676+++ juju/providers/common/tests/data/cloud_init_image 2012-11-23 21:00:27 +0000
677@@ -0,0 +1,35 @@
678+#cloud-config
679+apt_update: true
680+apt_upgrade: true
681+machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
682+ machine-id: passport}
683+output: {all: '| tee -a /var/log/cloud-init-output.log'}
684+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper]
685+runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf
686+ <<EOF
687+
688+ description "Juju machine agent"
689+
690+ author "Juju Team <juju@lists.ubuntu.com>"
691+
692+
693+ start on runlevel [2345]
694+
695+ stop on runlevel [!2345]
696+
697+ respawn
698+
699+
700+ env JUJU_MACHINE_ID="passport"
701+
702+ env JUJU_ZOOKEEPER="cotswold:2181,longleat:2181"
703+
704+
705+ exec python -m juju.agents.machine --nodaemon --logfile /var/log/juju/machine-agent.log
706+ --session-file /var/run/juju/machine-agent.zksession >> /tmp/juju-machine-agent.output
707+ 2>&1
708+
709+ EOF
710+
711+ ', /sbin/start juju-machine-agent]
712+ssh_authorized_keys: [chubb]
713
714=== added file 'juju/providers/common/tests/data/cloud_init_image_extra_env'
715--- juju/providers/common/tests/data/cloud_init_image_extra_env 1970-01-01 00:00:00 +0000
716+++ juju/providers/common/tests/data/cloud_init_image_extra_env 2012-11-23 21:00:27 +0000
717@@ -0,0 +1,39 @@
718+#cloud-config
719+apt_update: true
720+apt_upgrade: true
721+machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
722+ machine-id: passport}
723+output: {all: '| tee -a /var/log/cloud-init-output.log'}
724+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper]
725+runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf
726+ <<EOF
727+
728+ description "Juju machine agent"
729+
730+ author "Juju Team <juju@lists.ubuntu.com>"
731+
732+
733+ start on runlevel [2345]
734+
735+ stop on runlevel [!2345]
736+
737+ respawn
738+
739+
740+ env JUJU_MACHINE_ID="passport"
741+
742+ env JUJU_ZOOKEEPER="cotswold:2181,longleat:2181"
743+
744+ env http_proxy="http://myproxy.com:3128/"
745+
746+ env https_proxy="http://myproxy.com:3128/"
747+
748+
749+ exec python -m juju.agents.machine --nodaemon --logfile /var/log/juju/machine-agent.log
750+ --session-file /var/run/juju/machine-agent.zksession >> /tmp/juju-machine-agent.output
751+ 2>&1
752+
753+ EOF
754+
755+ ', /sbin/start juju-machine-agent]
756+ssh_authorized_keys: [chubb]
757
758=== modified file 'juju/providers/common/tests/data/cloud_init_no_machine_id'
759--- juju/providers/common/tests/data/cloud_init_no_machine_id 2012-09-24 04:04:32 +0000
760+++ juju/providers/common/tests/data/cloud_init_no_machine_id 2012-11-23 21:00:27 +0000
761@@ -4,7 +4,7 @@
762 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
763 machine-id: }
764 output: {all: '| tee -a /var/log/cloud-init-output.log'}
765-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws,
766- python-zookeeper, juju]
767+packages: [bzr, byobu, tmux, python-setuptools, python-twisted,
768+ python-zookeeper, juju, python-txaws]
769 runcmd: ['sudo mkdir -p /var/lib/juju', 'sudo mkdir -p /var/log/juju']
770 ssh_authorized_keys: [chubb]
771
772=== modified file 'juju/providers/common/tests/data/cloud_init_normal'
773--- juju/providers/common/tests/data/cloud_init_normal 2012-08-23 16:14:42 +0000
774+++ juju/providers/common/tests/data/cloud_init_normal 2012-11-23 21:00:27 +0000
775@@ -4,8 +4,8 @@
776 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
777 machine-id: passport}
778 output: {all: '| tee -a /var/log/cloud-init-output.log'}
779-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws,
780- python-zookeeper, juju]
781+packages: [bzr, byobu, tmux, python-setuptools, python-twisted,
782+ python-zookeeper, juju, python-txaws]
783 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir
784 -p /var/log/juju, 'JUJU_MACHINE_ID=passport JUJU_ZOOKEEPER=cotswold:2181,longleat:2181
785 python -m juju.agents.machine -n --logfile=/var/log/juju/machine-agent.log
786
787=== modified file 'juju/providers/common/tests/data/cloud_init_ppa'
788--- juju/providers/common/tests/data/cloud_init_ppa 2012-10-09 18:55:55 +0000
789+++ juju/providers/common/tests/data/cloud_init_ppa 2012-11-23 21:00:27 +0000
790@@ -6,8 +6,8 @@
791 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
792 machine-id: passport}
793 output: {all: '| tee -a /var/log/cloud-init-output.log'}
794-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
795- juju]
796+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
797+ juju, python-txaws]
798 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf
799 <<EOF
800
801
802=== modified file 'juju/providers/common/tests/data/cloud_init_ppa_apt_proxy'
803--- juju/providers/common/tests/data/cloud_init_ppa_apt_proxy 2012-10-09 18:55:55 +0000
804+++ juju/providers/common/tests/data/cloud_init_ppa_apt_proxy 2012-11-23 21:00:27 +0000
805@@ -7,8 +7,8 @@
806 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
807 machine-id: passport}
808 output: {all: '| tee -a /var/log/cloud-init-output.log'}
809-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
810- juju]
811+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
812+ juju, python-txaws]
813 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf
814 <<EOF
815
816
817=== modified file 'juju/providers/common/tests/data/cloud_init_proposed'
818--- juju/providers/common/tests/data/cloud_init_proposed 2012-08-23 16:14:42 +0000
819+++ juju/providers/common/tests/data/cloud_init_proposed 2012-11-23 21:00:27 +0000
820@@ -6,7 +6,7 @@
821 machine-data: {juju-provider-type: dummy, juju-zookeeper-hosts: 'cotswold:2181,longleat:2181',
822 machine-id: passport}
823 output: {all: '| tee -a /var/log/cloud-init-output.log'}
824-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper, juju]
825+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, juju, python-txaws]
826 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p
827 /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf <<EOF
828
829
830=== modified file 'juju/providers/common/tests/test_cloudinit.py'
831--- juju/providers/common/tests/test_cloudinit.py 2012-10-09 18:55:55 +0000
832+++ juju/providers/common/tests/test_cloudinit.py 2012-11-23 21:00:27 +0000
833@@ -107,6 +107,18 @@
834 "/usr/lib/pymodules/python2.7/juju/__init__.pyc")
835 self.assert_render(self.construct_normal(), "cloud_init_distro")
836
837+ def test_render_image_source(self):
838+ cloud_init = self.construct_normal()
839+ cloud_init.set_juju_source(image=True)
840+ self.assert_render(cloud_init, "cloud_init_image")
841+
842+ def test_render_extra_environment(self):
843+ cloud_init = self.construct_normal()
844+ cloud_init.set_juju_source(image=True)
845+ cloud_init.set_env("http_proxy", "http://myproxy.com:3128/")
846+ cloud_init.set_env("https_proxy", "http://myproxy.com:3128/")
847+ self.assert_render(cloud_init, "cloud_init_image_extra_env")
848+
849 def test_render_ppa_source(self):
850 cloud_init = self.construct_normal()
851 cloud_init.set_juju_source(ppa=True)
852@@ -135,6 +147,12 @@
853 def test_render_bootstrap(self):
854 self.assert_render(self.construct_bootstrap(), "cloud_init_bootstrap")
855
856+ def test_render_bootstrap_with_extra_environment(self):
857+ cloud_init = self.construct_bootstrap()
858+ cloud_init.set_env("http_proxy", "http://myproxy.com:3128/")
859+ cloud_init.set_env("https_proxy", "http://myproxy.com:3128/")
860+ self.assert_render(cloud_init, "cloud_init_bootstrap_extra_env")
861+
862 def test_render_bootstrap_with_zookeepers(self):
863 self.assert_render(
864 self.construct_bootstrap(True), "cloud_init_bootstrap_zookeepers")
865@@ -168,6 +186,13 @@
866 cloud_init.set_zookeeper_hosts(['bar1.private', 'bar2.private'])
867 self.assertEqual(['bar1.private', 'bar2.private'], cloud_init._zookeeper_hosts)
868
869+ def test_set_zookeeper_machines_in_private_mode(self):
870+ machine = ProviderMachine('i-100', 'foo1', 'foo1.private', private_ip_address='10.0.1.0')
871+ cloud_init = CloudInit()
872+ cloud_init.private = True
873+ cloud_init.set_zookeeper_machines([machine])
874+ self.assertEqual(['10.0.1.0'], cloud_init._zookeeper_hosts)
875+
876 def test_set_zookeeper_hosts_tuple(self):
877 cloud_init = CloudInit()
878 cloud_init.set_zookeeper_hosts([('bar1.private', 9999)])
879@@ -330,3 +355,14 @@
880 self.patch(juju, "__file__",
881 "/usr/lib/pymodules/python2.7/juju/__init__.pyc")
882 self.assertEquals(get_default_origin(), ("distro", None))
883+
884+ def test_set_env(self):
885+ cloudinit = CloudInit()
886+ cloudinit.set_env("http_proxy", "http://myproxy.com:3128/")
887+ self.assertEquals(cloudinit._extra_environment["http_proxy"], "http://myproxy.com:3128/")
888+
889+ def test_unset_env(self):
890+ cloudinit = CloudInit()
891+ cloudinit.set_env("http_proxy", "http://myproxy.com:3128/")
892+ cloudinit.unset_env("http_proxy")
893+ self.assertNotIn("http_proxy", cloudinit._extra_environment)
894
895=== modified file 'juju/providers/ec2/__init__.py'
896--- juju/providers/ec2/__init__.py 2012-08-28 16:11:41 +0000
897+++ juju/providers/ec2/__init__.py 2012-11-23 21:00:27 +0000
898@@ -149,12 +149,13 @@
899 % (", ".join(instance_ids), message))
900
901 machines = []
902+ private = self.config.get("vpc_id") is not None
903 for instance in instances:
904 if instance.instance_state not in ("running", "pending"):
905 continue
906- if group_name not in instance.reservation.groups:
907+ if group_name not in [g[1] for g in instance.reservation.groups]:
908 continue
909- machines.append(machine_from_instance(instance))
910+ machines.append(machine_from_instance(instance, private))
911
912 if instance_ids:
913 # We were asked for a specific list of machines, and if we can't
914
915=== modified file 'juju/providers/ec2/launch.py'
916--- juju/providers/ec2/launch.py 2012-07-05 21:49:12 +0000
917+++ juju/providers/ec2/launch.py 2012-11-23 21:00:27 +0000
918@@ -38,15 +38,20 @@
919 spec = yield get_machine_spec(self._provider.config, self._constraints)
920 security_groups = yield self._ensure_groups(machine_id)
921
922+ subnet = self._provider.config.get("subnet_id")
923+ kw = {
924+ 'min_count': 1,
925+ 'max_count': 1,
926+ 'image_id': spec.image_id,
927+ 'instance_type': spec.instance_type,
928+ 'availability_zone': availability_zone,
929+ 'user_data': user_data,
930+ }
931+ if subnet:
932+ kw['subnet_id'] = subnet
933+ kw['security_group_ids'] = security_groups
934 log.debug("Launching with machine spec %s", spec)
935- instances = yield self._provider.ec2.run_instances(
936- min_count=1,
937- max_count=1,
938- image_id=spec.image_id,
939- instance_type=spec.instance_type,
940- security_groups=security_groups,
941- availability_zone=availability_zone,
942- user_data=user_data)
943+ instances = yield self._provider.ec2.run_instances(**kw)
944
945 returnValue([machine_from_instance(i) for i in instances])
946
947@@ -68,18 +73,24 @@
948 self._provider.environment_name, machine_id)
949
950 security_groups = yield self._provider.ec2.describe_security_groups()
951- group_ids = [group.name for group in security_groups]
952+ group_names = [group.name for group in security_groups]
953+ vpc_id = self._provider.config.get("vpc_id")
954
955 # Create the provider group if doesn't exist.
956- if not juju_group in group_ids:
957+ if juju_group in group_names:
958+ for group in security_groups:
959+ if group.name == juju_group:
960+ juju_group_id = group.id
961+ else:
962 log.debug("Creating juju provider group %s", juju_group)
963- yield self._provider.ec2.create_security_group(
964+ juju_group_id = yield self._provider.ec2.create_security_group(
965 juju_group,
966- "juju group for %s" % self._provider.environment_name)
967+ "juju group for %s" % self._provider.environment_name,
968+ vpc_id=vpc_id)
969
970 # Authorize SSH.
971 yield self._provider.ec2.authorize_security_group(
972- juju_group,
973+ group_id=juju_group_id,
974 ip_protocol="tcp",
975 from_port="22", to_port="22",
976 cidr_ip="0.0.0.0/0")
977@@ -90,15 +101,20 @@
978
979 # Authorize Internal ZK Traffic
980 yield self._provider.ec2.authorize_security_group(
981- juju_group,
982+ group_id=juju_group_id,
983 source_group_name=juju_group,
984 source_group_owner_id=groups_info.pop().owner_id)
985
986 # Create the machine-specific group if it does not already exist
987- if not juju_machine_group in group_ids:
988- yield self._provider.ec2.create_security_group(
989+ if juju_machine_group in group_names:
990+ for group in security_groups:
991+ if group.name == juju_machine_group:
992+ juju_machine_group_id = group.id
993+ else:
994+ juju_machine_group_id = yield self._provider.ec2.create_security_group(
995 juju_machine_group,
996 "juju group for %s machine %s" % (
997- self._provider.environment_name, machine_id))
998+ self._provider.environment_name, machine_id),
999+ vpc_id=vpc_id)
1000
1001- returnValue([juju_group, juju_machine_group])
1002+ returnValue([juju_group_id, juju_machine_group_id])
1003
1004=== modified file 'juju/providers/ec2/machine.py'
1005--- juju/providers/ec2/machine.py 2011-10-11 10:59:04 +0000
1006+++ juju/providers/ec2/machine.py 2012-11-23 21:00:27 +0000
1007@@ -8,15 +8,19 @@
1008 """
1009
1010
1011-def machine_from_instance(instance):
1012+def machine_from_instance(instance, private=False):
1013 """Create an :class:`EC2ProviderMachine` from a txaws :class:`Instance`
1014
1015 :param instance: the EC2 Instance
1016
1017 :return: a matching :class:`EC2ProviderMachine`
1018 """
1019- return EC2ProviderMachine(
1020+ params = [
1021 instance.instance_id,
1022 instance.dns_name,
1023 instance.private_dns_name,
1024- instance.instance_state)
1025+ instance.instance_state,
1026+ instance.private_ip_address]
1027+ if private:
1028+ params[1] = params[2] = instance.private_ip_address
1029+ return EC2ProviderMachine(*params)
1030
1031=== modified file 'juju/providers/ec2/securitygroup.py'
1032--- juju/providers/ec2/securitygroup.py 2012-07-05 21:49:12 +0000
1033+++ juju/providers/ec2/securitygroup.py 2012-11-23 21:00:27 +0000
1034@@ -6,14 +6,32 @@
1035 from .utils import log
1036
1037
1038+@inlineCallbacks
1039+def _get_security_group_id(group_name, ec2):
1040+ groups = yield ec2.describe_security_groups()
1041+ for group in groups:
1042+ if group.name == group_name:
1043+ returnValue(group.id)
1044+
1045+
1046 def _get_juju_security_group(provider):
1047 """Get EC2 security group name for environment of `provider`."""
1048 return "juju-%s" % provider.environment_name
1049
1050
1051+def _get_juju_security_group_id(provider):
1052+ name = "juju-%s" % provider.environment_name
1053+ return _get_security_group_id(name, provider.ec2)
1054+
1055+
1056 def _get_machine_group_name(provider, machine_id):
1057+ return "juju-%s-%s" % (provider.environment_name, machine_id)
1058+
1059+
1060+def _get_machine_group_id(provider, machine_id):
1061 """Get EC2 security group name associated just with `machine_id`."""
1062- return "juju-%s-%s" % (provider.environment_name, machine_id)
1063+ name = _get_machine_group_name(provider, machine_id)
1064+ return _get_security_group_id(name, provider.ec2)
1065
1066
1067 # TODO These security group functions do not handle the eventual
1068@@ -34,8 +52,9 @@
1069 def open_provider_port(provider, machine, machine_id, port, protocol):
1070 """Authorize `port`/`proto` for the machine security group."""
1071 try:
1072+ group_id = yield _get_machine_group_id(provider, machine_id)
1073 yield provider.ec2.authorize_security_group(
1074- _get_machine_group_name(provider, machine_id),
1075+ group_id=group_id,
1076 ip_protocol=protocol,
1077 from_port=str(port), to_port=str(port),
1078 cidr_ip="0.0.0.0/0")
1079@@ -51,8 +70,9 @@
1080 def close_provider_port(provider, machine, machine_id, port, protocol):
1081 """Revoke `port`/`proto` for the machine security group."""
1082 try:
1083+ group_id = yield _get_machine_group_id(provider, machine_id)
1084 yield provider.ec2.revoke_security_group(
1085- _get_machine_group_name(provider, machine_id),
1086+ group_id=group_id,
1087 ip_protocol=protocol,
1088 from_port=str(port), to_port=str(port),
1089 cidr_ip="0.0.0.0/0")
1090@@ -73,15 +93,20 @@
1091 proto) pairs.
1092 """
1093 try:
1094- security_groups = yield provider.ec2.describe_security_groups(
1095- _get_machine_group_name(provider, machine_id))
1096+ security_group = None
1097+ security_groups = yield provider.ec2.describe_security_groups()
1098+ name = _get_machine_group_name(provider, machine_id)
1099+ for group in security_groups:
1100+ if group.name == name:
1101+ security_group = group
1102+ break
1103 except EC2Error, e:
1104 raise ProviderInteractionError(
1105 "Unexpected EC2Error getting open ports on machine %s: %s"
1106 % (machine.instance_id, e.get_error_messages()))
1107
1108 opened_ports = set() # made up of (port, protocol) pairs
1109- for ip_permission in security_groups[0].allowed_ips:
1110+ for ip_permission in security_group.allowed_ips:
1111 if ip_permission.cidr_ip != "0.0.0.0/0":
1112 continue
1113 from_port = int(ip_permission.from_port)
1114@@ -110,24 +135,15 @@
1115
1116
1117 @inlineCallbacks
1118-def _delete_security_group(provider, group):
1119- """Wrap EC2 delete_security_group."""
1120- try:
1121- yield provider.ec2.delete_security_group(group)
1122- log.debug("Deleted security group %r", group)
1123- except EC2Error, e:
1124- raise ProviderInteractionError(
1125- "EC2 error when attempting to delete group %s: %s" % (group, e))
1126-
1127-
1128-@inlineCallbacks
1129 def destroy_environment_security_group(provider):
1130 """Delete the security group for the environment of `provider`"""
1131 group = _get_juju_security_group(provider)
1132 try:
1133- yield provider.ec2.delete_security_group(group)
1134- log.debug("Deleted environment security group %r", group)
1135- returnValue(True)
1136+ group_id = yield _get_juju_security_group_id(provider)
1137+ if group_id:
1138+ yield provider.ec2.delete_security_group(id=group_id)
1139+ log.debug("Deleted environment security group %r", group)
1140+ returnValue(True)
1141 except EC2Error, e:
1142 # Ignore, since this is only attempting to cleanup
1143 log.debug(
1144
1145=== modified file 'juju/providers/ec2/tests/common.py'
1146--- juju/providers/ec2/tests/common.py 2012-09-10 03:20:20 +0000
1147+++ juju/providers/ec2/tests/common.py 2012-11-23 21:00:27 +0000
1148@@ -49,8 +49,8 @@
1149 def get_instance(self,
1150 instance_id, state="running", machine_id=42, **kwargs):
1151 groups = kwargs.pop("groups",
1152- ["juju-%s" % self.env_name,
1153- "juju-%s-%s" % (self.env_name, machine_id)])
1154+ [("sg-a1a1a1", "juju-%s" % self.env_name),
1155+ ("sg-b2b2b2", "juju-%s-%s" % (self.env_name, machine_id))])
1156 reservation = Reservation("x", "y", groups=groups)
1157 return Instance(instance_id, state, reservation=reservation, **kwargs)
1158
1159@@ -108,32 +108,33 @@
1160 get_ami(*get_ami_args)
1161 self.mocker.result(succeed(ami_name))
1162
1163- def _mock_create_group(self):
1164+ def _mock_create_group(self, vpc_id=None):
1165 group_name = "juju-%s" % self.env_name
1166 self.ec2.create_security_group(
1167- group_name, "juju group for %s" % self.env_name)
1168- self.mocker.result(succeed(True))
1169+ group_name, "juju group for %s" % self.env_name,
1170+ vpc_id=vpc_id)
1171+ self.mocker.result(succeed("sg-a1a1a1a1"))
1172
1173 self.ec2.authorize_security_group(
1174- group_name, ip_protocol="tcp", from_port="22",
1175+ group_id="sg-a1a1a1a1", ip_protocol="tcp", from_port="22",
1176 to_port="22", cidr_ip="0.0.0.0/0")
1177 self.mocker.result(succeed([self.env_name]))
1178
1179 self.ec2.describe_security_groups(group_name)
1180 self.mocker.result(succeed(
1181- [SecurityGroup(group_name, "", owner_id="123")]))
1182+ [SecurityGroup("sg-a1a1a1a1", group_name, "", owner_id="123")]))
1183
1184 self.ec2.authorize_security_group(
1185- group_name, source_group_name=group_name,
1186+ group_id="sg-a1a1a1a1", source_group_name=group_name,
1187 source_group_owner_id="123")
1188 self.mocker.result(succeed(True))
1189
1190- def _mock_create_machine_group(self, machine_id):
1191+ def _mock_create_machine_group(self, machine_id, vpc_id=None):
1192 machine_group_name = "juju-%s-%s" % (self.env_name, machine_id)
1193 self.ec2.create_security_group(
1194 machine_group_name, "juju group for %s machine %s" % (
1195- self.env_name, machine_id))
1196- self.mocker.result(succeed(True))
1197+ self.env_name, machine_id), vpc_id=vpc_id)
1198+ self.mocker.result(succeed("sg-b2b2b2b2"))
1199
1200 def _mock_get_zookeeper_hosts(self, hosts=None):
1201 """
1202@@ -147,7 +148,7 @@
1203
1204 if hosts is None:
1205 hosts = [self.get_instance(
1206- "i-es-zoo", private_dns_name="es.example.internal")]
1207+ "i-es-zoo", private_dns_name="es.example.internal", private_ip_address="1.1.1.1")]
1208
1209 self.s3.get_object(self.env_name, "provider-state")
1210 if hosts is False:
1211
1212=== modified file 'juju/providers/ec2/tests/data/bootstrap_cloud_init'
1213--- juju/providers/ec2/tests/data/bootstrap_cloud_init 2012-08-23 16:14:42 +0000
1214+++ juju/providers/ec2/tests/data/bootstrap_cloud_init 2012-11-23 21:00:27 +0000
1215@@ -3,8 +3,8 @@
1216 apt_upgrade: true
1217 machine-data: {juju-provider-type: ec2, juju-zookeeper-hosts: 'localhost:2181', machine-id: '0'}
1218 output: {all: '| tee -a /var/log/cloud-init-output.log'}
1219-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
1220- default-jre-headless, zookeeper, zookeeperd, juju]
1221+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
1222+ default-jre-headless, zookeeper, zookeeperd, juju, python-txaws]
1223 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'juju-admin initialize
1224 --instance-id=$(curl http://169.254.169.254/1.0/meta-data/instance-id) --admin-identity=admin:JbJ6sDGV37EHzbG9FPvttk64cmg=
1225 --constraints-data=e2NwdTogbnVsbCwgaW5zdGFuY2UtdHlwZTogbTEuc21hbGwsIG1lbTogbnVsbCwgcHJvdmlkZXItdHlwZTogZWMyLCB1YnVudHUtc2VyaWVzOiBzcGxlbmRpZH0K
1226
1227=== modified file 'juju/providers/ec2/tests/data/launch_cloud_init'
1228--- juju/providers/ec2/tests/data/launch_cloud_init 2012-08-23 16:14:42 +0000
1229+++ juju/providers/ec2/tests/data/launch_cloud_init 2012-11-23 21:00:27 +0000
1230@@ -4,7 +4,7 @@
1231 machine-data: {juju-provider-type: ec2, juju-zookeeper-hosts: 'es.example.internal:2181',
1232 machine-id: '1'}
1233 output: {all: '| tee -a /var/log/cloud-init-output.log'}
1234-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper, juju]
1235+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, juju, python-txaws]
1236 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p
1237 /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf <<EOF
1238
1239
1240=== modified file 'juju/providers/ec2/tests/data/launch_cloud_init_branch'
1241--- juju/providers/ec2/tests/data/launch_cloud_init_branch 2012-10-09 18:55:55 +0000
1242+++ juju/providers/ec2/tests/data/launch_cloud_init_branch 2012-11-23 21:00:27 +0000
1243@@ -6,7 +6,7 @@
1244 machine-data: {juju-provider-type: ec2, juju-zookeeper-hosts: 'es.example.internal:2181',
1245 machine-id: '1'}
1246 output: {all: '| tee -a /var/log/cloud-init-output.log'}
1247-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper]
1248+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, python-txaws]
1249 runcmd: [sudo apt-get install -y python-txzookeeper, sudo mkdir -p /usr/lib/juju,
1250 'cd /usr/lib/juju && sudo /usr/bin/bzr co --lightweight lp:~wizard/juju-juicebar juju', cd /usr/lib/juju/juju
1251 && sudo python setup.py develop, sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju,
1252
1253=== added file 'juju/providers/ec2/tests/data/launch_cloud_init_ip_address'
1254--- juju/providers/ec2/tests/data/launch_cloud_init_ip_address 1970-01-01 00:00:00 +0000
1255+++ juju/providers/ec2/tests/data/launch_cloud_init_ip_address 2012-11-23 21:00:27 +0000
1256@@ -0,0 +1,35 @@
1257+#cloud-config
1258+apt_update: true
1259+apt_upgrade: true
1260+machine-data: {juju-provider-type: ec2, juju-zookeeper-hosts: '1.1.1.1:2181',
1261+ machine-id: '1'}
1262+output: {all: '| tee -a /var/log/cloud-init-output.log'}
1263+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper, juju]
1264+runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p
1265+ /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf <<EOF
1266+
1267+ description "Juju machine agent"
1268+
1269+ author "Juju Team <juju@lists.ubuntu.com>"
1270+
1271+
1272+ start on runlevel [2345]
1273+
1274+ stop on runlevel [!2345]
1275+
1276+ respawn
1277+
1278+
1279+ env JUJU_MACHINE_ID="1"
1280+
1281+ env JUJU_ZOOKEEPER="1.1.1.1:2181"
1282+
1283+
1284+ exec python -m juju.agents.machine --nodaemon --logfile /var/log/juju/machine-agent.log
1285+ --session-file /var/run/juju/machine-agent.zksession >> /tmp/juju-machine-agent.output
1286+ 2>&1
1287+
1288+ EOF
1289+
1290+ ', /sbin/start juju-machine-agent]
1291+ssh_authorized_keys: [zebra]
1292
1293=== modified file 'juju/providers/ec2/tests/data/launch_cloud_init_ppa'
1294--- juju/providers/ec2/tests/data/launch_cloud_init_ppa 2012-10-09 18:55:55 +0000
1295+++ juju/providers/ec2/tests/data/launch_cloud_init_ppa 2012-11-23 21:00:27 +0000
1296@@ -6,7 +6,7 @@
1297 machine-data: {juju-provider-type: ec2, juju-zookeeper-hosts: 'es.example.internal:2181',
1298 machine-id: '1'}
1299 output: {all: '| tee -a /var/log/cloud-init-output.log'}
1300-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper, juju]
1301+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, juju, python-txaws]
1302 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p
1303 /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf <<EOF
1304
1305
1306=== modified file 'juju/providers/ec2/tests/test_bootstrap.py'
1307--- juju/providers/ec2/tests/test_bootstrap.py 2012-09-10 03:20:20 +0000
1308+++ juju/providers/ec2/tests/test_bootstrap.py 2012-11-23 21:00:27 +0000
1309@@ -48,7 +48,7 @@
1310 instance_type="m1.small",
1311 max_count=1,
1312 min_count=1,
1313- security_groups=["juju-moon", "juju-moon-0"],
1314+ security_group_ids=["sg-a1a1a1a1", "sg-b2b2b2b2"],
1315 availability_zone=None,
1316 user_data=MATCH(verify_user_data))
1317
1318@@ -90,7 +90,7 @@
1319 self._mock_verify()
1320 self.ec2.describe_security_groups()
1321 self.mocker.result(succeed([
1322- SecurityGroup("juju-%s" % self.env_name, "")]))
1323+ SecurityGroup("sg-a1a1a1a1", "juju-%s" % self.env_name, "")]))
1324 self._mock_create_machine_group(0)
1325 self._mock_launch_utils()
1326 self._mock_launch()
1327@@ -138,7 +138,7 @@
1328 self._mock_verify()
1329 self.ec2.describe_security_groups()
1330 self.mocker.result(succeed([
1331- SecurityGroup("juju-%s" % self.env_name, "")]))
1332+ SecurityGroup("sg-a1a1a1a1", "juju-%s" % self.env_name, "")]))
1333 self._mock_create_machine_group(0)
1334 self._mock_launch_utils()
1335 self._mock_launch()
1336
1337=== modified file 'juju/providers/ec2/tests/test_launch.py'
1338--- juju/providers/ec2/tests/test_launch.py 2012-09-10 03:20:20 +0000
1339+++ juju/providers/ec2/tests/test_launch.py 2012-11-23 21:00:27 +0000
1340@@ -31,7 +31,7 @@
1341 def _mock_launch(self, instance, expect_ami="ami-default",
1342 expect_instance_type="m1.small",
1343 expect_availability_zone=None,
1344- cloud_init="launch_cloud_init"):
1345+ cloud_init="launch_cloud_init", **extra):
1346
1347 def verify_user_data(data):
1348 expect_path = os.path.join(DATA_DIR, cloud_init)
1349@@ -45,12 +45,26 @@
1350 instance_type=expect_instance_type,
1351 max_count=1,
1352 min_count=1,
1353- security_groups=["juju-moon", "juju-moon-1"],
1354 availability_zone=expect_availability_zone,
1355- user_data=MATCH(verify_user_data))
1356+ user_data=MATCH(verify_user_data), **extra)
1357
1358 self.mocker.result(succeed([instance]))
1359
1360+ def _mock_launch_groupname(self, instance, expect_ami="ami-default",
1361+ expect_instance_type="m1.small",
1362+ expect_availability_zone=None,
1363+ cloud_init="launch_cloud_init", **extra):
1364+ extra["security_group_ids"] = ["sg-a1a1a1a1", "sg-b2b2b2b2"]
1365+ self._mock_launch(instance, expect_ami, expect_instance_type, expect_availability_zone, cloud_init, **extra)
1366+
1367+ def _mock_launch_subnet(self, instance, subnet_id, expect_ami="ami-default",
1368+ expect_instance_type="m1.small",
1369+ expect_availability_zone=None,
1370+ cloud_init="launch_cloud_init", **extra):
1371+ extra["subnet_id"] = subnet_id
1372+ extra["security_group_ids"] = ["sg-a1a1a1a1", "sg-b2b2b2b2"]
1373+ self._mock_launch(instance, expect_ami, expect_instance_type, expect_availability_zone, cloud_init, **extra)
1374+
1375 def test_bad_data(self):
1376 self.mocker.replay()
1377 d = self.get_provider().start_machine({})
1378@@ -74,13 +88,39 @@
1379 self._mock_create_machine_group("1")
1380 self._mock_launch_utils()
1381 self._mock_get_zookeeper_hosts()
1382- self._mock_launch(self.get_instance("i-foobar"))
1383- self.mocker.replay()
1384-
1385- def verify_result(result):
1386- (machine,) = result
1387- self.assert_machine(machine, "i-foobar", "")
1388- provider = self.get_provider()
1389+ self._mock_launch_groupname(self.get_instance("i-foobar"))
1390+ self.mocker.replay()
1391+
1392+ def verify_result(result):
1393+ (machine,) = result
1394+ self.assert_machine(machine, "i-foobar", "")
1395+ provider = self.get_provider()
1396+ d = provider.start_machine({
1397+ "machine-id": "1", "constraints": self.constraints})
1398+ d.addCallback(verify_result)
1399+ return d
1400+
1401+ def test_provider_launch_with_subnet_and_vpc(self):
1402+ vpc_id = "vpc-a1d4f2"
1403+ self.ec2.describe_security_groups()
1404+ self.mocker.result(succeed([]))
1405+ self._mock_create_group(vpc_id=vpc_id)
1406+ self._mock_create_machine_group("1", vpc_id=vpc_id)
1407+ self._mock_launch_utils()
1408+ self._mock_get_zookeeper_hosts()
1409+ self._mock_launch_subnet(
1410+ self.get_instance("i-foobar", private_ip_address="1.1.1.1"),
1411+ "subnet-a7f22c3e",
1412+ cloud_init="launch_cloud_init_ip_address"
1413+ )
1414+ self.mocker.replay()
1415+
1416+ def verify_result(result):
1417+ (machine,) = result
1418+ self.assert_machine(machine, "i-foobar", "")
1419+ provider = self.get_provider()
1420+ provider.config["subnet_id"] = "subnet-a7f22c3e"
1421+ provider.config["vpc_id"] = vpc_id
1422 d = provider.start_machine({
1423 "machine-id": "1", "constraints": self.constraints})
1424 d.addCallback(verify_result)
1425@@ -104,7 +144,7 @@
1426 self._mock_create_machine_group("1")
1427 self._mock_launch_utils()
1428 self._mock_get_zookeeper_hosts()
1429- self._mock_launch(
1430+ self._mock_launch_groupname(
1431 self.get_instance("i-foobar"),
1432 cloud_init="launch_cloud_init_branch")
1433 self.mocker.replay()
1434@@ -124,7 +164,7 @@
1435 self._mock_create_machine_group("1")
1436 self._mock_launch_utils()
1437 self._mock_get_zookeeper_hosts()
1438- self._mock_launch(
1439+ self._mock_launch_groupname(
1440 self.get_instance("i-foobar"),
1441 cloud_init="launch_cloud_init_ppa")
1442 self.mocker.replay()
1443@@ -144,7 +184,7 @@
1444 self._mock_create_machine_group("1")
1445 self._mock_launch_utils()
1446 self._mock_get_zookeeper_hosts()
1447- self._mock_launch(
1448+ self._mock_launch_groupname(
1449 self.get_instance("i-foobar"),
1450 cloud_init="launch_cloud_init")
1451 self.mocker.replay()
1452@@ -159,14 +199,15 @@
1453 def test_provider_launch_existing_security_group(self):
1454 """Verify that the launch works if the env security group exists"""
1455 instance = Instance("i-foobar", "running", dns_name="x1.example.com")
1456- security_group = SecurityGroup("juju-moon", "some description")
1457+ security_group = SecurityGroup("sg-a1a1a1a1", "juju-moon", "some description")
1458
1459 self.ec2.describe_security_groups()
1460 self.mocker.result(succeed([security_group]))
1461+ self.mocker.count(1)
1462 self._mock_create_machine_group("1")
1463 self._mock_launch_utils()
1464 self._mock_get_zookeeper_hosts()
1465- self._mock_launch(instance)
1466+ self._mock_launch_groupname(instance)
1467 self.mocker.replay()
1468
1469 provider = self.get_provider()
1470@@ -180,15 +221,13 @@
1471 def test_provider_launch_existing_machine_security_group(self):
1472 """Verify that the launch works if the machine security group exists"""
1473 instance = Instance("i-foobar", "running", dns_name="x1.example.com")
1474- machine_group = SecurityGroup(
1475- "juju-moon-1", "some description")
1476-
1477+ machine_group = SecurityGroup("sg-b2b2b2b2", "juju-moon-1", "some description")
1478 self.ec2.describe_security_groups()
1479 self.mocker.result(succeed([machine_group]))
1480 self._mock_create_group()
1481 self._mock_launch_utils()
1482 self._mock_get_zookeeper_hosts()
1483- self._mock_launch(instance)
1484+ self._mock_launch_groupname(instance)
1485 self.mocker.replay()
1486
1487 provider = self.get_provider()
1488@@ -206,15 +245,13 @@
1489 that security group, generally because it is still shutting
1490 down."""
1491 instance = Instance("i-foobar", "running", dns_name="x1.example.com")
1492- machine_group = SecurityGroup(
1493- "juju-moon-1", "some description")
1494-
1495+ machine_group = SecurityGroup("sg-b2b2b2b2", "juju-moon-1", "some description")
1496 self.ec2.describe_security_groups()
1497 self.mocker.result(succeed([machine_group]))
1498 self._mock_create_group()
1499 self._mock_launch_utils()
1500 self._mock_get_zookeeper_hosts()
1501- self._mock_launch(instance)
1502+ self._mock_launch_groupname(instance)
1503 self.mocker.replay()
1504
1505 provider = self.get_provider()
1506@@ -266,7 +303,7 @@
1507 get_ami_args=(
1508 "splendid", "amd64", "somewhere-else-1", False, False))
1509 self._mock_get_zookeeper_hosts()
1510- self._mock_launch(self.get_instance("i-foobar"), "ami-regional")
1511+ self._mock_launch_groupname(self.get_instance("i-foobar"), "ami-regional")
1512 self.mocker.replay()
1513
1514 provider = self.get_provider()
1515@@ -283,7 +320,7 @@
1516 ami_name="ami-fancy-cluster",
1517 get_ami_args=("vast", "amd64", "us-east-1", True, False))
1518 self._mock_get_zookeeper_hosts()
1519- self._mock_launch(
1520+ self._mock_launch_groupname(
1521 self.get_instance("i-foobar"), "ami-fancy-cluster",
1522 expect_instance_type="cc2.8xlarge",
1523 expect_availability_zone="us-east-1b")
1524@@ -302,7 +339,7 @@
1525 self._mock_launch_utils(
1526 get_ami_args=("dribbly", "amd64", "us-east-1", False, False))
1527 self._mock_get_zookeeper_hosts()
1528- self._mock_launch(
1529+ self._mock_launch_groupname(
1530 self.get_instance("i-foobar"), "ami-default",
1531 expect_instance_type="c1.xlarge")
1532 self.mocker.replay()
1533
1534=== modified file 'juju/providers/ec2/tests/test_machine.py'
1535--- juju/providers/ec2/tests/test_machine.py 2011-10-11 14:03:11 +0000
1536+++ juju/providers/ec2/tests/test_machine.py 2012-11-23 21:00:27 +0000
1537@@ -11,7 +11,8 @@
1538 instance = Instance(
1539 "i-foobar", "oscillating",
1540 dns_name="public",
1541- private_dns_name="private")
1542+ private_dns_name="private",
1543+ private_ip_address="10.0.1.10")
1544
1545 machine = machine_from_instance(instance)
1546 self.assertTrue(isinstance(machine, EC2ProviderMachine))
1547@@ -19,3 +20,19 @@
1548 self.assertEquals(machine.dns_name, "public")
1549 self.assertEquals(machine.private_dns_name, "private")
1550 self.assertEquals(machine.state, "oscillating")
1551+ self.assertEquals(machine.private_ip_address, "10.0.1.10")
1552+
1553+ def test_machine_from_instance_private_mode(self):
1554+ instance = Instance(
1555+ "i-foobar", "oscillating",
1556+ dns_name="public",
1557+ private_dns_name="private",
1558+ private_ip_address="10.0.1.10")
1559+
1560+ machine = machine_from_instance(instance, True)
1561+ self.assertTrue(isinstance(machine, EC2ProviderMachine))
1562+ self.assertEquals(machine.instance_id, "i-foobar")
1563+ self.assertEquals(machine.dns_name, "10.0.1.10")
1564+ self.assertEquals(machine.private_dns_name, "10.0.1.10")
1565+ self.assertEquals(machine.state, "oscillating")
1566+ self.assertEquals(machine.private_ip_address, "10.0.1.10")
1567
1568=== modified file 'juju/providers/ec2/tests/test_securitygroup.py'
1569--- juju/providers/ec2/tests/test_securitygroup.py 2012-07-05 21:49:12 +0000
1570+++ juju/providers/ec2/tests/test_securitygroup.py 2012-11-23 21:00:27 +0000
1571@@ -18,9 +18,12 @@
1572 def test_open_provider_port(self):
1573 """Verify open port op will use the correct EC2 API."""
1574 log = self.capture_logging("juju.ec2", level=logging.DEBUG)
1575+ secgroup = SecurityGroup("sg-a1a1a1", "juju-moon-machine-1", "some description")
1576+ self.ec2.describe_security_groups()
1577+ self.mocker.result(succeed([secgroup]))
1578 machine = ProviderMachine("i-foobar", "x1.example.com")
1579 self.ec2.authorize_security_group(
1580- "juju-moon-machine-1", ip_protocol="tcp", from_port="80",
1581+ group_id=secgroup.id, ip_protocol="tcp", from_port="80",
1582 to_port="80", cidr_ip="0.0.0.0/0")
1583 self.mocker.result(succeed(True))
1584 self.mocker.replay()
1585@@ -35,9 +38,12 @@
1586 def test_close_provider_port(self):
1587 """Verify close port op will use the correct EC2 API."""
1588 log = self.capture_logging("juju.ec2", level=logging.DEBUG)
1589+ secgroup = SecurityGroup("sg-b2b2b2", "juju-moon-machine-1", "some description")
1590+ self.ec2.describe_security_groups()
1591+ self.mocker.result(succeed([secgroup]))
1592 machine = ProviderMachine("i-foobar", "x1.example.com")
1593 self.ec2.revoke_security_group(
1594- "juju-moon-machine-1", ip_protocol="tcp", from_port="80",
1595+ group_id="sg-b2b2b2", ip_protocol="tcp", from_port="80",
1596 to_port="80", cidr_ip="0.0.0.0/0")
1597 self.mocker.result(succeed(True))
1598 self.mocker.replay()
1599@@ -51,9 +57,10 @@
1600 @inlineCallbacks
1601 def test_get_provider_opened_ports(self):
1602 """Verify correct parse of IP perms from describe_security_group."""
1603- self.ec2.describe_security_groups("juju-moon-machine-1")
1604+ self.ec2.describe_security_groups()
1605 self.mocker.result(succeed([
1606 SecurityGroup(
1607+ "sg-a1b2c3d4e5f6",
1608 "juju-%s-machine-1" % self.env_name,
1609 "a security group name",
1610 ips=[
1611@@ -76,9 +83,12 @@
1612 @inlineCallbacks
1613 def test_open_provider_port_unknown_instance(self):
1614 """Verify open port op will use the correct EC2 API."""
1615+ secgroup = SecurityGroup("sg-a1a1a1", "juju-moon-machine-1", "some description")
1616+ self.ec2.describe_security_groups()
1617+ self.mocker.result(succeed([secgroup]))
1618 machine = ProviderMachine("i-foobar", "x1.example.com")
1619 self.ec2.authorize_security_group(
1620- "juju-moon-machine-1", ip_protocol="tcp", from_port="80",
1621+ group_id="sg-a1a1a1", ip_protocol="tcp", from_port="80",
1622 to_port="80", cidr_ip="0.0.0.0/0")
1623 self.mocker.result(fail(self.get_ec2_error("i-foobar")))
1624 self.mocker.replay()
1625@@ -95,9 +105,12 @@
1626 @inlineCallbacks
1627 def test_close_provider_port_unknown_instance(self):
1628 """Verify open port op will use the correct EC2 API."""
1629+ secgroup = SecurityGroup("sg-b2b2b2", "juju-moon-machine-1", "some description")
1630+ self.ec2.describe_security_groups()
1631+ self.mocker.result(succeed([secgroup]))
1632 machine = ProviderMachine("i-foobar", "x1.example.com")
1633 self.ec2.revoke_security_group(
1634- "juju-moon-machine-1", ip_protocol="tcp", from_port="80",
1635+ group_id="sg-b2b2b2", ip_protocol="tcp", from_port="80",
1636 to_port="80", cidr_ip="0.0.0.0/0")
1637 self.mocker.result(fail(self.get_ec2_error("i-foobar")))
1638 self.mocker.replay()
1639@@ -114,7 +127,7 @@
1640 @inlineCallbacks
1641 def test_get_provider_opened_ports_unknown_instance(self):
1642 """Verify open port op will use the correct EC2 API."""
1643- self.ec2.describe_security_groups("juju-moon-machine-1")
1644+ self.ec2.describe_security_groups()
1645 self.mocker.result(fail(self.get_ec2_error("i-foobar")))
1646 self.mocker.replay()
1647
1648@@ -134,7 +147,10 @@
1649 @inlineCallbacks
1650 def test_destroy_environment_security_group(self):
1651 """Verify the deletion of the security group for the environment"""
1652- self.ec2.delete_security_group("juju-moon")
1653+ secgroup = SecurityGroup("sg-0e0e0e", "juju-moon", "some description")
1654+ self.ec2.describe_security_groups()
1655+ self.mocker.result(succeed([secgroup]))
1656+ self.ec2.delete_security_group(id="sg-0e0e0e")
1657 self.mocker.result(succeed(True))
1658 self.mocker.replay()
1659
1660@@ -143,10 +159,12 @@
1661 self.assertTrue(destroyed)
1662
1663 @inlineCallbacks
1664- def test_destroy_environment_security_group_missing(self):
1665- """Verify ignores errors in deleting the env security group"""
1666+ def test_destroy_environment_security_group_ignores_failure(self):
1667 log = self.capture_logging(level=logging.DEBUG)
1668- self.ec2.delete_security_group("juju-moon")
1669+ secgroup = SecurityGroup("sg-0e0e0e", "juju-moon", "some description")
1670+ self.ec2.describe_security_groups()
1671+ self.mocker.result(succeed([secgroup]))
1672+ self.ec2.delete_security_group(id="sg-0e0e0e")
1673 self.mocker.result(fail(
1674 self.get_ec2_error(
1675 "juju-moon",
1676@@ -162,3 +180,14 @@
1677 "juju-moon: Error Message: The security group "
1678 "'juju-moon' does not exist",
1679 log.getvalue())
1680+
1681+ @inlineCallbacks
1682+ def test_destroy_environment_security_group_missing(self):
1683+ """Verify ignores errors in deleting the env security group"""
1684+ self.ec2.describe_security_groups()
1685+ self.mocker.result(succeed([]))
1686+ self.mocker.replay()
1687+
1688+ provider = self.get_provider()
1689+ destroyed = yield destroy_environment_security_group(provider)
1690+ self.assertFalse(destroyed)
1691
1692=== modified file 'juju/providers/ec2/tests/test_shutdown.py'
1693--- juju/providers/ec2/tests/test_shutdown.py 2012-07-09 22:59:18 +0000
1694+++ juju/providers/ec2/tests/test_shutdown.py 2012-11-23 21:00:27 +0000
1695@@ -2,12 +2,13 @@
1696
1697 from juju.lib.testing import TestCase
1698 from juju.providers.ec2.tests.common import (
1699- EC2TestMixin, MATCH_GROUP, Observed, MockInstanceState)
1700+ EC2TestMixin, MATCH_GROUP, MockInstanceState)
1701
1702 from juju.machine import ProviderMachine
1703
1704 from juju.errors import MachinesNotFound, ProviderError
1705 from juju.providers.ec2.machine import EC2ProviderMachine
1706+from txaws.ec2.model import SecurityGroup
1707
1708
1709 class SomeError(Exception):
1710@@ -156,10 +157,11 @@
1711 ("i-canbekilled", "running", "shutting-down"),
1712 ("i-canbekilledtoo", "pending", "shutting-down")]))
1713
1714- self.ec2.delete_security_group(MATCH_GROUP)
1715- deleted_groups = Observed()
1716- self.mocker.call(deleted_groups.add)
1717- self.mocker.count(1)
1718+ secgroup = SecurityGroup("sg-0a0a0a", "juju-moon", "some description")
1719+ self.ec2.describe_security_groups()
1720+ self.mocker.result(succeed([secgroup]))
1721+ self.ec2.delete_security_group(id="sg-0a0a0a")
1722+ self.mocker.result(succeed(True))
1723
1724 self.mocker.replay()
1725
1726@@ -169,9 +171,6 @@
1727 self.assertEquals(machine_1.instance_id, "i-canbekilled")
1728 self.assertTrue(isinstance(machine_2, EC2ProviderMachine))
1729 self.assertEquals(machine_2.instance_id, "i-canbekilledtoo")
1730- self.assertEquals(
1731- deleted_groups.items,
1732- set(["juju-moon"]))
1733
1734 @inlineCallbacks
1735 def test_s3_failure(self):
1736@@ -189,7 +188,10 @@
1737 self.ec2.terminate_instances("i-canbekilled")
1738 self.mocker.result(succeed([
1739 ("i-canbekilled", "running", "shutting-down")]))
1740- self.ec2.delete_security_group("juju-moon")
1741+ secgroup = SecurityGroup("sg-0a0a0a", "juju-moon", "some description")
1742+ self.ec2.describe_security_groups()
1743+ self.mocker.result(succeed([secgroup]))
1744+ self.ec2.delete_security_group(id="sg-0a0a0a")
1745 self.mocker.result(succeed(True))
1746 self.mocker.replay()
1747
1748@@ -208,7 +210,10 @@
1749 self.mocker.result(succeed(None))
1750 self.ec2.describe_instances()
1751 self.mocker.result(succeed([]))
1752- self.ec2.delete_security_group("juju-moon")
1753+ secgroup = SecurityGroup("sg-0a0a0a", "juju-moon", "some description")
1754+ self.ec2.describe_security_groups()
1755+ self.mocker.result(succeed([secgroup]))
1756+ self.ec2.delete_security_group(id="sg-0a0a0a")
1757 self.mocker.result(fail(
1758 self.get_ec2_error(
1759 "juju-moon",
1760
1761=== modified file 'juju/providers/ec2/utils.py'
1762--- juju/providers/ec2/utils.py 2012-07-05 21:49:12 +0000
1763+++ juju/providers/ec2/utils.py 2012-11-23 21:00:27 +0000
1764@@ -6,10 +6,10 @@
1765 import StringIO
1766
1767 from twisted.internet.defer import inlineCallbacks, returnValue
1768-from twisted.web.client import getPage
1769 from twisted.web.error import Error
1770
1771 from juju.errors import ProviderError
1772+from juju.lib.http_client import get_page
1773
1774 # We don't actually know what's available in any given region
1775 _PLAUSIBLE_ZONES = ascii_lowercase
1776@@ -102,7 +102,7 @@
1777 contextFactory = VerifyingContextFactory(_CURRENT_IMAGE_HOST)
1778 else:
1779 contextFactory = None
1780- d = getPage(uri, contextFactory=contextFactory)
1781+ d = get_page(uri, contextFactory=contextFactory)
1782 d.addErrback(handle_404)
1783 d.addCallback(extract_ami)
1784 return d
1785
1786=== modified file 'juju/providers/local/files.py'
1787--- juju/providers/local/files.py 2012-09-28 06:13:47 +0000
1788+++ juju/providers/local/files.py 2012-11-23 21:00:27 +0000
1789@@ -3,10 +3,10 @@
1790
1791 from twisted.internet.defer import inlineCallbacks, returnValue
1792 from twisted.internet.error import ConnectionRefusedError
1793-from twisted.web.client import getPage
1794
1795 from juju.errors import ProviderError, FileNotFound
1796 from juju.lib import serializer
1797+from juju.lib.http_client import get_page
1798 from juju.lib.service import TwistedDaemonService
1799 from juju.providers.common.files import FileStorage
1800
1801@@ -55,7 +55,7 @@
1802 def is_serving(self):
1803 try:
1804 storage = LocalStorage(self._storage_dir)
1805- yield getPage((yield storage.get_url(SERVER_URL_KEY)))
1806+ yield get_page((yield storage.get_url(SERVER_URL_KEY)))
1807 returnValue(True)
1808 except ConnectionRefusedError:
1809 returnValue(False)
1810
1811=== modified file 'juju/providers/orchestra/files.py'
1812--- juju/providers/orchestra/files.py 2011-10-17 07:43:09 +0000
1813+++ juju/providers/orchestra/files.py 2012-11-23 21:00:27 +0000
1814@@ -2,10 +2,10 @@
1815 import urllib
1816 import urlparse
1817
1818-from twisted.web.client import getPage
1819 from twisted.web.error import Error
1820
1821 from juju.errors import FileNotFound, ProviderError
1822+from juju.lib.http_client import get_page
1823 from juju.providers.common.utils import convert_unknown_error
1824 from juju.providers.orchestra.digestauth import (
1825 DigestAuthenticator, get_page_auth)
1826@@ -60,7 +60,7 @@
1827 :raises: :exc:`juju.errors.FileNotFound` if the file doesn't exist
1828 """
1829 url = self.get_url(name)
1830- d = getPage(url)
1831+ d = get_page(url)
1832 d.addCallback(StringIO)
1833 d.addErrback(_convert_error, "GET", url, {404: FileNotFound(url)})
1834 return d
1835
1836=== modified file 'juju/providers/orchestra/tests/data/bootstrap_user_data'
1837--- juju/providers/orchestra/tests/data/bootstrap_user_data 2012-08-23 16:14:42 +0000
1838+++ juju/providers/orchestra/tests/data/bootstrap_user_data 2012-11-23 21:00:27 +0000
1839@@ -3,8 +3,8 @@
1840 machine-data: {juju-provider-type: orchestra, juju-zookeeper-hosts: 'localhost:2181',
1841 machine-id: '0'}
1842 output: {all: '| tee -a /var/log/cloud-init-output.log'}
1843-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper,
1844- default-jre-headless, zookeeper, zookeeperd, juju]
1845+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper,
1846+ default-jre-headless, zookeeper, zookeeperd, juju, python-txaws]
1847 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p /var/log/juju, 'juju-admin initialize
1848 --instance-id=winston-uid --admin-identity=admin:qRBXC1ubEEUqRL6wcBhgmc9xkaY=
1849 --constraints-data=e29yY2hlc3RyYS1jbGFzc2VzOiAnZm9vLGJhcicsIHByb3ZpZGVyLXR5cGU6IG9yY2hlc3RyYSwgdWJ1bnR1LXNlcmllczogYml6YXJyZX0K
1850
1851=== modified file 'juju/providers/orchestra/tests/data/launch_user_data'
1852--- juju/providers/orchestra/tests/data/launch_user_data 2012-08-23 16:14:42 +0000
1853+++ juju/providers/orchestra/tests/data/launch_user_data 2012-11-23 21:00:27 +0000
1854@@ -3,7 +3,7 @@
1855 machine-data: {juju-provider-type: orchestra, juju-zookeeper-hosts: 'jennifer:2181',
1856 machine-id: '42'}
1857 output: {all: '| tee -a /var/log/cloud-init-output.log'}
1858-packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-txaws, python-zookeeper, juju]
1859+packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-zookeeper, juju, python-txaws]
1860 runcmd: [sudo mkdir -p /var/lib/juju, sudo mkdir -p
1861 /var/log/juju, 'cat >> /etc/init/juju-machine-agent.conf <<EOF
1862
1863
1864=== modified file 'juju/unit/address.py'
1865--- juju/unit/address.py 2012-10-05 17:49:27 +0000
1866+++ juju/unit/address.py 2012-11-23 21:00:27 +0000
1867@@ -1,5 +1,6 @@
1868 """Service units have both a public and private address.
1869 """
1870+import os
1871 import subprocess
1872
1873 from twisted.internet.defer import inlineCallbacks, returnValue, succeed
1874@@ -15,7 +16,9 @@
1875 settings = GlobalSettingsStateManager(client)
1876 provider_type = yield settings.get_provider_type()
1877 if provider_type == "ec2":
1878- returnValue(EC2UnitAddress())
1879+ address = EC2UnitAddress()
1880+ address.private = os.environ.get("JUJU_PRIVATE") is not None
1881+ returnValue(address)
1882 if provider_type in ("openstack", "openstack_s3"):
1883 returnValue(OpenStackUnitAddress())
1884 elif provider_type == "local":
1885@@ -50,6 +53,10 @@
1886
1887 class EC2UnitAddress(UnitAddress):
1888
1889+ def __init__(self, *args, **kwargs):
1890+ super(EC2UnitAddress, self).__init__(*args, **kwargs)
1891+ self.private = False
1892+
1893 @inlineCallbacks
1894 def get_private_address(self):
1895 content = yield client.getPage(
1896@@ -58,8 +65,12 @@
1897
1898 @inlineCallbacks
1899 def get_public_address(self):
1900+ if self.private:
1901+ field = "local-ipv4"
1902+ else:
1903+ field = "public-hostname"
1904 content = yield client.getPage(
1905- "http://169.254.169.254/latest/meta-data/public-hostname")
1906+ "http://169.254.169.254/latest/meta-data/%s" % field)
1907 returnValue(content.strip())
1908
1909
1910
1911=== modified file 'juju/unit/charm.py'
1912--- juju/unit/charm.py 2012-03-21 12:54:28 +0000
1913+++ juju/unit/charm.py 2012-11-23 21:00:27 +0000
1914@@ -3,12 +3,12 @@
1915 import shutil
1916
1917 from twisted.internet.defer import inlineCallbacks, returnValue
1918-from twisted.web.client import downloadPage
1919 from twisted.web.error import Error
1920
1921 from juju.errors import FileNotFound
1922 from juju.charm.bundle import CharmBundle
1923 from juju.lib import under
1924+from juju.lib.http_client import download_page
1925 from juju.state.charm import CharmStateManager
1926
1927
1928@@ -33,7 +33,7 @@
1929 shutil.copyfileobj(open(file_path), open(local_charm_path, "w"))
1930 else:
1931 try:
1932- yield downloadPage(charm_state.bundle_url, local_charm_path)
1933+ yield download_page(charm_state.bundle_url, local_charm_path)
1934 except Error:
1935 raise FileNotFound(charm_state.bundle_url)
1936
1937
1938=== modified file 'juju/unit/deploy.py'
1939--- juju/unit/deploy.py 2012-04-04 20:21:44 +0000
1940+++ juju/unit/deploy.py 2012-11-23 21:00:27 +0000
1941@@ -80,6 +80,11 @@
1942 deployment = self.deploy_factory(
1943 service_unit_name, self.juju_directory)
1944
1945+ for env in ("http_proxy", "https_proxy", "JUJU_PRIVATE"):
1946+ value = os.environ.get(env)
1947+ if value:
1948+ deployment.set_env(env, value)
1949+
1950 log.debug("Using %r for %s in %s",
1951 deployment,
1952 service_unit_name,
1953
1954=== modified file 'juju/unit/tests/test_address.py'
1955--- juju/unit/tests/test_address.py 2012-10-05 17:49:27 +0000
1956+++ juju/unit/tests/test_address.py 2012-11-23 21:00:27 +0000
1957@@ -1,3 +1,4 @@
1958+import os
1959 import subprocess
1960 import zookeeper
1961
1962@@ -30,6 +31,14 @@
1963 def test_get_ec2_address(self):
1964 address = yield self.get_address_for("ec2")
1965 self.assertTrue(isinstance(address, EC2UnitAddress))
1966+ self.assertFalse(address.private)
1967+
1968+ @inlineCallbacks
1969+ def test_get_ec2_address_private_mode(self):
1970+ os.environ.setdefault("JUJU_PRIVATE", "private")
1971+ address = yield self.get_address_for("ec2")
1972+ self.assertTrue(address.private)
1973+ del os.environ["JUJU_PRIVATE"]
1974
1975 @inlineCallbacks
1976 def test_get_openstack_address(self):
1977@@ -118,6 +127,23 @@
1978 self.assertEqual(
1979 (yield self.address.get_public_address()), "foobar")
1980
1981+ @inlineCallbacks
1982+ def test_get_address_private_mode(self):
1983+ urls = [
1984+ "http://169.254.169.254/latest/meta-data/local-hostname",
1985+ "http://169.254.169.254/latest/meta-data/local-ipv4"]
1986+
1987+ def verify_args(url):
1988+ self.assertEqual(urls.pop(0), url)
1989+ return succeed("foobar\n")
1990+
1991+ self.address.private = True
1992+ self.patch(client, "getPage", verify_args)
1993+ self.assertEqual(
1994+ (yield self.address.get_private_address()), "foobar")
1995+ self.assertEqual(
1996+ (yield self.address.get_public_address()), "foobar")
1997+ self.address.private = False
1998
1999 class OpenStackAddressTest(TestCase):
2000

Subscribers

People subscribed via source and target branches

to status/vote changes: