Merge lp:~makyo/juju-deployer/machines-and-placement into lp:juju-deployer

[Madison Scott-Clary]

Add v4 support for machine spec per the bundles spec.

QA:
Ensure that each of the bundles in deployer/tests/test_data/v4/ deploy via:
  PYTHONPATH=. python deployer/cli.py -c deployer/tests/test_data/v4/<bundle>.yaml

All tests should pass.

[Brad Crittenden (bac)]

QA instructions with PYTHONPATH won't work if juju-deployer has previously been pip installed.

Still doing QA.

[Francesco Banconi (frankban)]

Hi Madison, this is an impressive branch.
Thank you for sorting out the intricacies of the new units placement logic.
Did not review the tests yet, but I have some suggestions and questions below: please let me know what do you think.
I am not sure about repeated placement parsing logic across the modules in this branch (see below).
I'll take another and deeper look after your replies.
Thank you!

[David Britton (davidpbritton)]

Hi -- Other than examining the code, where can I see differences between the v4 and v3 spec? Could a README change be introduced with perhaps an example v4 file with annotated differences? Also, rather than duplicating the entire v3 class, could things be refactored to be shared, and then tested together? Like a common set of validations, and then just the v4 additions (same for 'get()').

I have a number of comments inline, mostly around test coverage. I found one coding error just by inspection, making me a bit worried about the lack of unit tests for the change. Relying on just the more functional style of tests (of parsing a complete bundle) for covering this new code seems like a risky approach.

Some of the more complex functions (ex: validate()) could also use from refactoring and breaking apart as they are multiple pages in length (with clear duplication in the v4 addition).

review: Needs Fixing

[Madison Scott-Clary (makyo)]

Checkpoint for pairing.

[Brad Crittenden (bac)]

Looks good Madison with a few comments.

[Madison Scott-Clary (makyo)]

For live environment testing, assuming an LXC env named local,

TEST_ENDPOINT=localhost JUJU_ENV=local nosetests --verbosity=2 deployer.tests.test_goenv:LiveEnvironmentTest

[Madison Scott-Clary (makyo)]

> For live environment testing, assuming an LXC env named local,
>
> TEST_ENDPOINT=localhost JUJU_ENV=local nosetests --verbosity=2
> deployer.tests.test_goenv:LiveEnvironmentTest

Also, ensure that that happens on a single line. LP broke it up

[Kapil Thangavelu (hazmat)]

What happens when i use a v4 spec with multiple machines, remove a unit that was placed to one of those machines, and then try to run it again? the code while it looks nice seems to have some gaps around consideration of running it multiple times, which sort of defines an ideal production usage (run again to apply delta from vcs).

[Francesco Banconi (frankban)]

Thanks for the fixes Madison, nicely done!
Also thanks for improving the pre-existing code docstrings and test coverage.
I have some minor questions and suggestions, please see below.

In the case Kapil described, the current behavior seems to be that we get the last unit placements.
In theory, we should really diff and retrieve the missing placement.
For the time being, a safe tradeoff could be to always create new machines for missing units if cur_units > 0.

[Madison Scott-Clary (makyo)]

> What happens when i use a v4 spec with multiple machines, remove a unit that
> was placed to one of those machines, and then try to run it again? the code
> while it looks nice seems to have some gaps around consideration of running it
> multiple times, which sort of defines an ideal production usage (run again to
> apply delta from vcs).

We talked about this with the team this morning (and there was additional discussion online last night), noting that this is something that the new bundle specification does not currently have in place. As part of this branch, I will change the logic that is used to to decide what a v4 bundle is (that is, only one with a machine spec), and all other bundles will fall back to the v3 format, which can be run multiple times.

[David Britton (davidpbritton)]

Thanks Makyo -- I have added a number of diff comments. Getting much better, thanks for the improvements and additional tests, though it could still use more. I will keep this as needs fixing, and I'll also test it out with a couple v3 bundles live while I wait to hear your response.

I'd still also like to see an addtion to the README pointing to or explaining the v4 spec, and an annotated file that highlights differences.

Thanks!

review: Needs Fixing

[Madison Scott-Clary (makyo)]

Committing most fixes now; tests will be in the next commit.

[lp:~makyo/juju-deployer/machines-and-placement]

153. By Madison Scott-Clary on 2015-03-18

Merge trunk; update tests for subordinate placements.

154. By Madison Scott-Clary on 2015-03-24

Add invalid subordinate placement v4 bundle.

155. By Madison Scott-Clary on 2015-03-24

Add series to machine spec.

[🤖 Landscape Builder (landscape-builder)]

Thanks, test coverage much better, follow-on branch will address concerns about attempts being idempotent. Deployed both v3 and v4 bundles with placement.

review: Approve

[David Britton (davidpbritton)]

Thanks, test coverage much better, follow-on branch will address concerns about attempts being idempotent. Deployed both v3 and v4 bundles with placement.

review: Approve

=== modified file 'README'
--- README	2013-07-24 12:46:56 +0000
+++ README	2015-03-24 17:39:12 +0000
@@ -21,7 +21,7 @@
 Stack Definitions
 -----------------
 
-High level view::
+High level view of v3 stacks::
 
   blog:
      series: precise
@@ -57,6 +57,10 @@
  We've got two deployment stacks here, blog, and blog-prod. The blog stack defines
  a simple wordpress deploy with mysql and two relations. In this case its
 
+Version 4 bundles are currently under development.  The development document for
+these types of bundles is available `here
+<https://docs.google.com/a/canonical.com/document/d/1SF8hTBi6oVbki8V__beNij6wnQU-5cm6PZsy5gf0j_Y>`_.
+
 
 Development
 -----------

=== modified file 'deployer/action/importer.py'
--- deployer/action/importer.py	2014-10-01 10:18:36 +0000
+++ deployer/action/importer.py	2015-03-24 17:39:12 +0000
@@ -15,6 +15,7 @@
         self.options = options
         self.env = env
         self.deployment = deployment
+        self.machines = {}
 
     def add_units(self):
         self.log.debug("Adding units...")
@@ -55,6 +56,50 @@
             else:
                 self.env.add_units(svc.name, abs(delta))
 
+    def create_machines(self):
+        """Create machines as specified in the machine spec in the bundle.
+
+        A machine spec consists of a named machine (the name is, by convention,
+        an integer) with an optional series, optional constraints and optional
+        annotations:
+
+        0:
+          series: "precise"
+          constraints: "mem=4G arch=i386"
+          annotations:
+            foo: bar
+        1:
+          constraints: "mem=4G"
+        
+        This method first attempts to create any machines in the 'machines'
+        section of the bundle specification with the given constraints and
+        annotations.  Then, if there are any machines requested for containers
+        in the style of <container type>:new, it creates those machines and
+        adds them to the machines map."""
+        machines = self.deployment.get_machines()
+        if machines:
+            self.log.info("Creating machines...")
+            for machine_name, spec in machines.items():
+                self.machines[machine_name] = self.env.add_machine(
+                        series=spec.get('series', 
+                                        self.deployment.data['series']),
+                        constraints=spec.get('constraints'))
+                annotations = spec.get('annotations')
+                if annotations:
+                    self.env.set_annotation(
+                        self.machines[machine_name],
+                        annotations,
+                        entity_type='machine')
+            # In the case of <container type>:new, we need to create a machine
+            # before creating the container on which the service will be
+            # deployed.  This is stored in the machines map which will be used
+            # in the service placement.
+            for service in self.deployment.get_services():
+                placement = self.deployment.get_unit_placement(service, None)
+                for container_host in placement.get_new_machines_for_containers():
+                    self.machines[container_host] = self.env.add_machine()
+            self.deployment.set_machines(machines)
+
     def get_charms(self):
         # Get Charms
         self.log.debug("Getting charms...")
@@ -66,7 +111,14 @@
         # validate them.
         self.deployment.resolve(self.options.overrides or ())
 
-    def deploy_services(self):
+    def deploy_services(self, add_units=True):
+        """Deploy the services specified in the deployment.
+
+        add_units: whether or not to add units to the service as it is
+          deployed; newer versions of bundles may have machines specified
+          in a machine spec, and units will be placed accordingly if this
+          flag is false.
+        """
         self.log.info("Deploying services...")
         self.log.debug(self.env)
         env_status = self.env.status()
@@ -84,17 +136,31 @@
 
             if svc.unit_placement:
                 # We sorted all the non placed services first, so we only
-                # need to update status once after we're done with them.
+                # need to update status once after we're done with them, in
+                # the instance of v3 bundles; in the more complex case of v4
+                # bundles, we'll need to refresh each time.
                 if not reloaded:
                     self.log.debug(
                         " Refetching status for placement deploys")
                     time.sleep(5.1)
                     env_status = self.env.status()
-                    reloaded = True
+                    # In the instance of version 3 deployments, we will not
+                    # need to fetch the status more than once.  In version 4
+                    # bundles, however, we will need to fetch the status each
+                    # time in order to allow for the machine specification to
+                    # be taken into account.
+                    if self.deployment.version == 3:
+                        reloaded = True
                 num_units = 1
             else:
                 num_units = svc.num_units
 
+            # Unset num_units if we are not adding units.  This should be done
+            # after the above work to ensure that the status is still retrieved
+            # as necessary.
+            if not add_units:
+                num_units = None
+
             placement = self.deployment.get_unit_placement(svc, env_status)
 
             if charm.is_subordinate():
@@ -193,7 +259,11 @@
             self.env.bootstrap()
         self.env.connect()
 
-        self.deploy_services()
+        if self.deployment.version > 3:
+            self.create_machines()
+
+        # We can shortcut and add the units during deployment for v3 bundles.
+        self.deploy_services(add_units=(self.deployment.version == 3))
 
         # Workaround api issue in juju-core, where any action takes 5s
         # to be consistent to subsequent watch api interactions, see

=== modified file 'deployer/config.py'
--- deployer/config.py	2015-02-24 15:49:28 +0000
+++ deployer/config.py	2015-03-24 17:39:12 +0000
@@ -52,7 +52,8 @@
 
         # Check if this is a v4 bundle.
         if 'services' in yaml_result:
-            self.version = 4
+            if 'machines' in yaml_result:
+                self.version = 4
             yaml_result = {config_file: yaml_result}
 
         self.yaml[config_file] = yaml_result
@@ -68,13 +69,14 @@
                              key, ", ".join(self.keys()))
             raise ErrorExit()
         deploy_data = self.data[key]
-        if self.version != 4:
+        if self.version < 4:
             deploy_data = self._resolve_inherited(deploy_data)
         if self.cli_series:
             deploy_data['series'] = self.cli_series
         return Deployment(
             key, deploy_data, self.include_dirs,
-            repo_path=os.environ.get("JUJU_REPOSITORY", ""))
+            repo_path=os.environ.get("JUJU_REPOSITORY", ""),
+            version=self.version)
 
     def load(self):
         data = {}

=== modified file 'deployer/deployment.py'
--- deployer/deployment.py	2015-02-06 21:43:24 +0000
+++ deployer/deployment.py	2015-03-24 17:39:12 +0000
@@ -7,20 +7,22 @@
 
 from .charm import Charm
 from .feedback import Feedback
-from .service import Service, ServiceUnitPlacement
+from .service import Service, ServiceUnitPlacementV3, ServiceUnitPlacementV4
 from .relation import Endpoint
-from .utils import path_join, yaml_dump, ErrorExit, resolve_include
+from .utils import path_join, yaml_dump, ErrorExit, resolve_include, x_in_y
 
 
 class Deployment(object):
 
     log = logging.getLogger("deployer.deploy")
 
-    def __init__(self, name, data, include_dirs, repo_path=""):
+    def __init__(self, name, data, include_dirs, repo_path="", version=3):
         self.name = name
         self.data = data
         self.include_dirs = include_dirs
         self.repo_path = repo_path
+        self.version = version
+        self.machines = {}
 
     @property
     def series(self):
@@ -44,17 +46,53 @@
         services = []
         for name, svc_data in self.data.get('services', {}).items():
             services.append(Service(name, svc_data))
-        services.sort(self._placement_sort)
+        if self.version == 3:
+            # Sort unplaced units first, then sort by name for placed units.
+            services.sort(key=lambda svc: (bool(svc.unit_placement), svc.name))
+        else:
+            services.sort(self._machines_placement_sort)
         return services
 
+    def set_machines(self, machines):
+        """Set a dict of machines, mapping from the names in the machine spec
+        to the machine names in the environment status.
+        """
+        self.machines = machines
+
+    def get_machines(self):
+        """Return a dict mapping machine names to machine options.
+
+        An empty dict is returned if no machines are defined in the
+        bundle YAML.
+        """
+        machines = {}
+        for key, machine in self.data.get('machines', {}).items():
+            machines[str(key)] = machine
+        return machines
+
     def get_service_names(self):
         """Return a sequence of service names for this deployment."""
         return self.data.get('services', {}).keys()
 
     @staticmethod
-    def _placement_sort(svc_a, svc_b):
+    def _machines_placement_sort(svc_a, svc_b):
+        """Sort machines with machine placement in mind.
+        
+        If svc_a is colocated alongside svc_b, svc_b needs to be deployed
+        first, so that it exists by the time svc_a is deployed, and vice
+        versa; this sorts first based on this fact, then secondly based on
+        whether or not the service has a unit placement, and then finally
+        based on the name of the service.
+        """
         if svc_a.unit_placement:
             if svc_b.unit_placement:
+                # Check for colocation.  This naively assumes that there is no
+                # circularity in placements.
+                if x_in_y(svc_b, svc_a):
+                    return -1
+                if x_in_y(svc_a, svc_b):
+                    return 1
+                # If no colocation exists, simply compare names.
                 return cmp(svc_a.name, svc_b.name)
             return 1
         if svc_b.unit_placement:
@@ -64,7 +102,11 @@
     def get_unit_placement(self, svc, status):
         if isinstance(svc, (str, unicode)):
             svc = self.get_service(svc)
-        return ServiceUnitPlacement(svc, self, status)
+        if self.version == 3:
+            return ServiceUnitPlacementV3(svc, self, status)
+        else:
+            return ServiceUnitPlacementV4(svc, self, status,
+                                          machines_map=self.machines)
 
     def get_relations(self):
         if 'relations' not in self.data:

=== modified file 'deployer/env/base.py'
--- deployer/env/base.py	2015-03-12 21:37:23 +0000
+++ deployer/env/base.py	2015-03-24 17:39:12 +0000
@@ -158,5 +158,5 @@
     def add_unit(self, service_name, machine_spec):
         raise NotImplementedError()
 
-    def set_annotation(self, entity, annotations):
+    def set_annotation(self, entity, annotations, entity_type='service'):
         raise NotImplementedError()

=== modified file 'deployer/env/go.py'
--- deployer/env/go.py	2014-09-16 17:04:46 +0000
+++ deployer/env/go.py	2015-03-24 17:39:12 +0000
@@ -1,7 +1,10 @@
 import time
 
 from .base import BaseEnvironment
-from ..utils import ErrorExit
+from ..utils import (
+    ErrorExit,
+    parse_constraints,
+)
 
 from jujuclient import (
     EnvError,
@@ -24,6 +27,20 @@
         self.api_endpoint = endpoint
         self.client = None
 
+    def add_machine(self, series="",constraints={}):
+        """Add a top level machine to the Juju environment.
+
+        Use the given series and constraints.
+        Return the machine identifier (e.g. "1").
+        
+        series: a string such as 'precise' or 'trusty'.
+        constraints: a map of constraints (such as mem, arch, etc.) which
+          can be parsed by utils.parse_constraints
+        """
+        return self.client.add_machine(
+            series=series, 
+            constraints=parse_constraints(constraints))['Machine']
+
     def add_unit(self, service_name, machine_spec):
         return self.client.add_unit(service_name, machine_spec)
 
@@ -196,8 +213,15 @@
             else:
                 return
 
-    def set_annotation(self, svc_name, annotation):
-        return self.client.set_annotation(svc_name, 'service', annotation)
+    def set_annotation(self, entity_name, annotation, entity_type='service'):
+        """Set an annotation on an entity.
+
+        entity_name: the name of the entity (machine, service, etc.) to
+          annotate.
+        annotation: a dict of key/value pairs to set on the entity.
+        entity_type: the type of entity (machine, service, etc.) to annotate.
+        """
+        return self.client.set_annotation(entity_name, entity_type, annotation)
 
     def status(self):
         return self.client.get_stat()

=== modified file 'deployer/service.py'
--- deployer/service.py	2015-03-13 17:41:18 +0000
+++ deployer/service.py	2015-03-24 17:39:12 +0000
@@ -1,3 +1,5 @@
+import itertools
+
 from feedback import Feedback
 
 
@@ -63,6 +65,57 @@
         else:
             return machine
 
+    def colocate(self, status, placement, u_idx, container, svc):
+        """Colocate one service with an existing service either within a
+        container alongside that service or hulk-smashed onto the same unit.
+
+        status: the environment status.
+        placement: the placement directive of the unit to be colocated.
+        u_idx: the unit index of the unit to be colocated.
+        container: a string containing the container type, or None.
+        svc: the service object for this placement.
+        """
+        with_service = status['services'].get(placement)
+        if with_service is None:
+            # Should be caught in validate relations but sanity check
+            # for concurrency.
+            self.deployment.log.error(
+                "Service %s to be deployed with non-existent service %s",
+                svc.name, placement)
+            # Prefer continuing deployment with a new machine rather
+            # than an in-progress abort.
+            return None
+
+        svc_units = with_service['units']
+        if int(u_idx) >= len(svc_units):
+            self.deployment.log.warning(
+                "Service:%s, Deploy-with-service:%s, Requested-unit-index=%s, "
+                "Cannot solve, falling back to default placement",
+                svc.name, placement, u_idx)
+            return None
+        unit_names = svc_units.keys()
+        unit_names.sort()
+        machine = svc_units[unit_names[int(u_idx)]].get('machine')
+        if not machine:
+            self.deployment.log.warning(
+                "Service:%s deploy-with unit missing machine %s",
+                svc.name, unit_names[int(u_idx)])
+            return None
+        return self._format_placement(machine, container)
+
+
+class ServiceUnitPlacementV3(ServiceUnitPlacement):
+
+    def _parse_placement(self, unit_placement):
+        placement = unit_placement
+        container = None
+        u_idx = None
+        if ':' in unit_placement:
+            container, placement = unit_placement.split(":")
+        if '=' in placement:
+            placement, u_idx = placement.split("=")
+        return container, placement, u_idx
+
     def validate(self):
         feedback = Feedback()
 
@@ -77,14 +130,13 @@
         services = dict([(s.name, s) for s in self.deployment.get_services()])
 
         for idx, p in enumerate(unit_placement):
-            if ':' in p:
-                container, p = p.split(':')
+            container, p, u_idx = self._parse_placement(p)
+            if container:
                 if container not in ('lxc', 'kvm'):
                     feedback.error(
-                        "Invalid service:%s placement: %s" % (
-                            self.service.name, unit_placement[idx]))
-            if '=' in p:
-                p, u_idx = p.split("=")
+                        "Invalid container type:%s service: %s placement: %s" \
+                            % (container, self.service.name, unit_placement[idx]))
+            if u_idx:
                 if p in ('maas', 'zone'):
                     continue
                 if not u_idx.isdigit():
@@ -117,6 +169,10 @@
         return feedback
 
     def get(self, unit_number):
+        """Get the placement directive for a given unit.
+
+        unit_number: the number of the unit to deploy
+        """
         status = self.status
         svc = self.service
 
@@ -143,30 +199,190 @@
         elif placement == "zone":
             return "zone=%s" % u_idx
 
-        with_service = status['services'].get(placement)
-        if with_service is None:
-            # Should be caught in validate relations but sanity check
-            # for concurrency.
-            self.deployment.log.error(
-                "Service %s to be deployed with non existant service %s",
-                svc.name, placement)
-            # Prefer continuing deployment with a new machine rather
-            # than an in-progress abort.
-            return None
-
-        svc_units = with_service['units']
-        if int(u_idx) >= len(svc_units):
-            self.deployment.log.warning(
-                "Service:%s, Deploy-with-service:%s, Requested-unit-index=%s, "
-                "Cannot solve, falling back to default placement",
-                svc.name, placement, u_idx)
-            return None
-        unit_names = svc_units.keys()
-        unit_names.sort()
-        machine = svc_units[unit_names[int(u_idx)]].get('machine')
-        if not machine:
-            self.deployment.log.warning(
-                "Service:%s deploy-with unit missing machine %s",
-                svc.name, unit_names[unit_number])
-            return None
-        return self._format_placement(machine, container)
+        return self.colocate(status, placement, u_idx, container, svc)
+
+
+class ServiceUnitPlacementV4(ServiceUnitPlacement):
+
+    def __init__(self, service, deployment, status, arbitrary_machines=False,
+                 machines_map=None):
+        super(ServiceUnitPlacementV4, self).__init__(
+            service, deployment, status, arbitrary_machines=arbitrary_machines)
+
+        # Arbitrary machines will not be allowed in v4 bundles.
+        self.arbitrary_machines = False
+
+        self.machines_map = machines_map
+
+        # Ensure that placement spec is filled according to the bundle
+        # specification.
+        self._fill_placement()
+
+    def _fill_placement(self):
+        """Fill the placement spec with necessary data.
+        
+        From the spec:
+        A unit placement may be specified with a service name only, in which
+        case its unit number is assumed to be one more than the unit number of
+        the previous unit in the list with the same service, or zero if there
+        were none.
+
+        If there are less elements in To than NumUnits, the last element is
+        replicated to fill it. If there are no elements (or To is omitted),
+        "new" is replicated.
+        """
+        unit_mapping = self.service.unit_placement
+        unit_count = self.service.num_units
+        if not unit_mapping:
+            self.service.svc_data['to'] = ['new'] * unit_count
+            return
+
+        self.service.svc_data['to'] = (
+            unit_mapping + 
+            list(itertools.repeat(unit_mapping[-1], unit_count - len(unit_mapping)))
+        )
+        unit_mapping = self.service.unit_placement
+
+        colocate_counts = {}
+        for idx, mapping in enumerate(unit_mapping):
+            service = mapping
+            if ':' in mapping:
+                service = mapping.split(':')[1]
+            if service in self.deployment.data['services']:
+                unit_number = colocate_counts.setdefault(service, 0)
+                unit_mapping[idx] = "{}/{}".format(mapping, unit_number)
+                colocate_counts[service] += 1
+        self.service.svc_data['to'] = unit_mapping
+
+    def _parse_placement(self, placement):
+        """Parse a unit placement statement.
+        
+        In version 4 bundles, unit placement statements take the form of
+        
+          (<containertype>:)?(<unit>|<machine>|new)
+
+        This splits the placement into a container, a placement, and a unit
+        number.  Both container and unit number are optional and can be None.
+        """
+        container = unit_number = None
+        if ':' in placement:
+            container, placement = placement.split(':')
+        if '/' in placement:
+            placement, unit_number = placement.split('/')
+        return container, placement, unit_number
+
+    def validate(self):
+        """Validate the placement of a service and all of its units.
+
+        If a service has a 'to' block specified, the list of machines, units,
+        containers, and/or services must be internally consistent, consistent
+        with other services in the deployment, and consistent with any machines
+        specified in the 'machines' block of the deployment.
+        
+        A feedback object is returned, potentially with errors and warnings
+        inside it.
+        """
+        feedback = Feedback()
+
+        unit_placement = self.service.unit_placement
+        if unit_placement is None:
+            return feedback
+
+        if not isinstance(unit_placement, (list, tuple)):
+            unit_placement = [unit_placement]
+        unit_placement = map(str, unit_placement)
+
+        services = dict([(s.name, s) for s in self.deployment.get_services()])
+        machines = self.deployment.get_machines()
+        container = None
+        unit_number = None
+
+        for i, placement in enumerate(unit_placement):
+            container, placement, unit_number = self._parse_placement(placement)
+
+            if container and container not in ('lxc', 'kvm'):
+                feedback.error(
+                    "Invalid container type: %s service: %s placement: %s" \
+                        % (container, self.service.name, unit_placement[i]))
+            # XXX Nesting containers not supported yet.
+            # Makyo - 2015-03-01
+            if container is not None and not (placement.isdigit()
+                    or placement == 'new'):
+                feedback.error(
+                    "Invalid target for container: %s" % (
+                        unit_placement[i]))
+            # Specify an existing machine (or, if the number is in the
+            # list of machine specs, one of those).
+            if placement.isdigit():
+                if placement in machines:
+                    continue
+                else:
+                    feedback.error(
+                        ("Service placement to machine "
+                         "not supported %s to %s") % (
+                             self.service.name, unit_placement[i]))
+            # Specify a machine from the machine spec.
+            elif placement in self.deployment.get_machines():
+                continue
+            # Specify a service for colocation.
+            elif placement in services:
+                # Specify a particular unit for colocation.
+                if unit_number is not None and \
+                        unit_number > services[placement].num_units:
+                    feedback.error(
+                        "Service unit does not exist, %s to %s/%s" % (
+                            self.service.name, placement, unit_number))
+                elif self.deployment.get_charm_for(placement).is_subordinate():
+                    feedback.error(
+                        "Cannot place to a subordinate service: %s -> %s" % (
+                            self.service.name, placement))
+            # Create a new machine or container.
+            elif placement == 'new':
+                continue
+            else:
+                feedback.error(
+                        "Invalid service placement %s to %s" % (
+                        self.service.name, unit_placement[i]))
+        return feedback
+
+    def get_new_machines_for_containers(self):
+        """Return a list of containers in the service's unit placement that
+        have been requested to be put on new machines."""
+        new_machines = []
+        unit = itertools.count()
+        for placement in self.service.unit_placement:
+            if ':new' in placement:
+                # Generate a name for this machine to be used in the
+                # machines_map used later; as a quick path forward, simply use
+                # the unit's name.
+                new_machines.append('{}/{}'.format(self.service.name, unit.next()))
+        return new_machines
+
+    def get(self, unit_number):
+        """Get the placement directive for a given unit.
+
+        unit_number: the number of the unit to deploy
+        """
+        status = self.status
+        svc = self.service
+
+        unit_mapping = svc.unit_placement
+        unit_placement = placement = str(unit_mapping[unit_number])
+        container = None
+        u_idx = unit_number
+
+        # Shortcut for new machines.
+        if placement == 'new':
+            return None
+
+        container, placement, unit_number = self._parse_placement(unit_placement)
+
+        if placement in self.machines_map:
+            return self._format_placement(self.machines_map[placement], container)
+
+        # Handle <container_type>:new
+        if placement == 'new':
+            return self._format_placement(
+                self.machines_map['%s/%d' % (self.service.name, u_idx)], container)
+
+        return self.colocate(status, placement, u_idx, container, svc)

=== modified file 'deployer/tests/base.py'
--- deployer/tests/base.py	2015-03-17 17:38:26 +0000
+++ deployer/tests/base.py	2015-03-24 17:39:12 +0000
@@ -30,13 +30,19 @@
     def tearDownClass(cls):
         shutil.rmtree(os.environ["JUJU_HOME"])
 
-    def get_named_deployment(self, file_name, stack_name):
-        """ Get deployment from test_data file.
+    def get_named_deployment_v3(self, file_name, stack_name):
+        """ Get v3 deployment from a test_data file.
         """
         return ConfigStack(
             [os.path.join(
                 self.test_data_dir, file_name)]).get(stack_name)
 
+    def get_deployment_v4(self, file_name):
+        """Get v4 deployment from a test_data file.
+        """
+        f = os.path.join(self.test_data_dir, 'v4', file_name)
+        return ConfigStack([f]).get(f)
+
     def capture_logging(self, name="", level=logging.INFO,
                         log_file=None, formatter=None):
         if log_file is None:

=== modified file 'deployer/tests/test_config.py'
--- deployer/tests/test_config.py	2015-02-26 17:33:21 +0000
+++ deployer/tests/test_config.py	2015-03-24 17:39:12 +0000
@@ -44,11 +44,11 @@
 
     def test_config_v4(self):
         config = ConfigStack([
-            os.path.join(self.test_data_dir, 'blog_v4.yaml')])
+            os.path.join(self.test_data_dir, 'v4', 'simple.yaml')])
         config.load()
         self.assertEqual(
             config.keys(),
-            [os.path.join(self.test_data_dir, 'blog_v4.yaml')])
+            [os.path.join(self.test_data_dir, 'v4', 'simple.yaml')])
         with mock.patch('deployer.config.ConfigStack._resolve_inherited') \
                 as mock_resolve:
             deployment = config.get(config.keys()[0])

=== added directory 'deployer/tests/test_data/v4'
=== added file 'deployer/tests/test_data/v4/container-new.yaml'
--- deployer/tests/test_data/v4/container-new.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/container-new.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,43 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    options:
+      debug: false
+      name: Please set name of wiki
+      skin: vector
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+    to:
+      - "1"
+  mysql:
+    charm: cs:precise/mysql-28
+    num_units: 1
+    options:
+      binlog-format: MIXED
+      block-size: 5
+      dataset-size: 80%
+      flavor: distro
+      ha-bindiface: eth0
+      ha-mcastport: 5411
+      max-connections: -1
+      preferred-storage-engine: InnoDB
+      query-cache-size: -1
+      query-cache-type: "OFF"
+      rbd-name: mysql1
+      tuning-level: safest
+      vip_cidr: 24
+      vip_iface: eth0
+    annotations:
+      gui-x: "610"
+      gui-y: "255"
+    to:
+      - "lxc:new"
+series: precise
+relations:
+- - mediawiki:db
+  - mysql:db
+machines:
+  1:
+    constraints: 'mem=512M'

=== added file 'deployer/tests/test_data/v4/container.yaml'
--- deployer/tests/test_data/v4/container.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/container.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,43 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    options:
+      debug: false
+      name: Please set name of wiki
+      skin: vector
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+    to:
+      - "1"
+  mysql:
+    charm: cs:precise/mysql-28
+    num_units: 1
+    options:
+      binlog-format: MIXED
+      block-size: 5
+      dataset-size: 80%
+      flavor: distro
+      ha-bindiface: eth0
+      ha-mcastport: 5411
+      max-connections: -1
+      preferred-storage-engine: InnoDB
+      query-cache-size: -1
+      query-cache-type: "OFF"
+      rbd-name: mysql1
+      tuning-level: safest
+      vip_cidr: 24
+      vip_iface: eth0
+    annotations:
+      gui-x: "610"
+      gui-y: "255"
+    to:
+      - "lxc:1"
+series: precise
+relations:
+- - mediawiki:db
+  - mysql:db
+machines:
+  1:
+    constraints: "mem=512M"

=== added file 'deployer/tests/test_data/v4/fill_placement.yaml'
--- deployer/tests/test_data/v4/fill_placement.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/fill_placement.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,17 @@
+services:
+  mediawiki1:
+    charm: cs:precise/mediawiki-10
+    num_units: 2
+  mediawiki2:
+    charm: cs:precise/mediawiki-10
+    num_units: 2
+    to:
+      - 0
+  mediawiki3:
+    charm: cs:precise/mediawiki-10
+    num_units: 2
+    to:
+      - mediawiki1
+      - mediawiki1
+machines: {}
+series: precise

=== added file 'deployer/tests/test_data/v4/hulk-smash-nounits-nomachines.yaml'
--- deployer/tests/test_data/v4/hulk-smash-nounits-nomachines.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/hulk-smash-nounits-nomachines.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,38 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    options:
+      debug: false
+      name: Please set name of wiki
+      skin: vector
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+  mysql:
+    charm: cs:precise/mysql-28
+    num_units: 1
+    options:
+      binlog-format: MIXED
+      block-size: 5
+      dataset-size: 80%
+      flavor: distro
+      ha-bindiface: eth0
+      ha-mcastport: 5411
+      max-connections: -1
+      preferred-storage-engine: InnoDB
+      query-cache-size: -1
+      query-cache-type: "OFF"
+      rbd-name: mysql1
+      tuning-level: safest
+      vip_cidr: 24
+      vip_iface: eth0
+    annotations:
+      gui-x: "610"
+      gui-y: "255"
+    to:
+      - "mediawiki"
+series: precise
+relations:
+- - mediawiki:db
+  - mysql:db

=== added file 'deployer/tests/test_data/v4/hulk-smash-nounits.yaml'
--- deployer/tests/test_data/v4/hulk-smash-nounits.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/hulk-smash-nounits.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,43 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    options:
+      debug: false
+      name: Please set name of wiki
+      skin: vector
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+    to:
+      - "1"
+  mysql:
+    charm: cs:precise/mysql-28
+    num_units: 1
+    options:
+      binlog-format: MIXED
+      block-size: 5
+      dataset-size: 80%
+      flavor: distro
+      ha-bindiface: eth0
+      ha-mcastport: 5411
+      max-connections: -1
+      preferred-storage-engine: InnoDB
+      query-cache-size: -1
+      query-cache-type: "OFF"
+      rbd-name: mysql1
+      tuning-level: safest
+      vip_cidr: 24
+      vip_iface: eth0
+    annotations:
+      gui-x: "610"
+      gui-y: "255"
+    to:
+      - "mediawiki"
+series: precise
+relations:
+- - mediawiki:db
+  - mysql:db
+machines:
+  1:
+    constraints: 'mem=512M'

=== added file 'deployer/tests/test_data/v4/hulk-smash.yaml'
--- deployer/tests/test_data/v4/hulk-smash.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/hulk-smash.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,43 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    options:
+      debug: false
+      name: Please set name of wiki
+      skin: vector
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+    to:
+      - "1"
+  mysql:
+    charm: cs:precise/mysql-28
+    num_units: 1
+    options:
+      binlog-format: MIXED
+      block-size: 5
+      dataset-size: 80%
+      flavor: distro
+      ha-bindiface: eth0
+      ha-mcastport: 5411
+      max-connections: -1
+      preferred-storage-engine: InnoDB
+      query-cache-size: -1
+      query-cache-type: "OFF"
+      rbd-name: mysql1
+      tuning-level: safest
+      vip_cidr: 24
+      vip_iface: eth0
+    annotations:
+      gui-x: "610"
+      gui-y: "255"
+    to:
+      - "mediawiki/0"
+series: precise
+relations:
+- - mediawiki:db
+  - mysql:db
+machines:
+  1:
+    constraints: 'mem=512M'

=== added file 'deployer/tests/test_data/v4/placement-invalid-subordinate.yaml'
--- deployer/tests/test_data/v4/placement-invalid-subordinate.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/placement-invalid-subordinate.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,13 @@
+series: precise
+services:
+  nova-compute:
+    charm: cs:precise/nova-compute
+    to:
+      - nrpe
+  ceph:
+    charm: cs:precise/ceph
+    to:
+      - lxc:nrpe/0
+  nrpe:
+    charm: cs:precise/nrpe
+    units: 2

=== added file 'deployer/tests/test_data/v4/placement.yaml'
--- deployer/tests/test_data/v4/placement.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/placement.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,45 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    options:
+      debug: false
+      name: Please set name of wiki
+      skin: vector
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+    to:
+      - "1"
+  mysql:
+    charm: cs:precise/mysql-28
+    num_units: 1
+    options:
+      binlog-format: MIXED
+      block-size: 5
+      dataset-size: 80%
+      flavor: distro
+      ha-bindiface: eth0
+      ha-mcastport: 5411
+      max-connections: -1
+      preferred-storage-engine: InnoDB
+      query-cache-size: -1
+      query-cache-type: "OFF"
+      rbd-name: mysql1
+      tuning-level: safest
+      vip_cidr: 24
+      vip_iface: eth0
+    annotations:
+      gui-x: "610"
+      gui-y: "255"
+    to:
+      - "2"
+series: precise
+relations:
+- - mediawiki:db
+  - mysql:db
+machines:
+  1:
+    constraints: 'mem=512M'
+  2:
+    constraints: 'mem=512M'

=== added file 'deployer/tests/test_data/v4/series.yaml'
--- deployer/tests/test_data/v4/series.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/series.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,21 @@
+services:
+  mediawiki:
+    charm: cs:precise/mediawiki-10
+    num_units: 1
+    annotations:
+      gui-x: "609"
+      gui-y: "-15"
+    to:
+      - "1"
+  mysql:
+    charm: cs:trusty/mysql-1
+    num_units: 1
+    to:
+      - "2"
+series: trusty
+machines:
+  1:
+    series: precise
+    constraints: 'mem=512M'
+  2:
+    constraints: 'mem=512M'

=== renamed file 'deployer/tests/test_data/blog_v4.yaml' => 'deployer/tests/test_data/v4/simple.yaml'
--- deployer/tests/test_data/blog_v4.yaml	2015-02-20 19:18:58 +0000
+++ deployer/tests/test_data/v4/simple.yaml	2015-03-24 17:39:12 +0000
@@ -30,6 +30,7 @@
     annotations:
       gui-x: "610"
       gui-y: "255"
+machines: {}
 series: precise
 relations:
 - - mediawiki:db

=== added file 'deployer/tests/test_data/v4/validate.yaml'
--- deployer/tests/test_data/v4/validate.yaml	1970-01-01 00:00:00 +0000
+++ deployer/tests/test_data/v4/validate.yaml	2015-03-24 17:39:12 +0000
@@ -0,0 +1,16 @@
+services:
+  mysql:
+    charm: 'cs:precise/mysql-1'
+    num_units: 5
+    to:
+      - 'asdf:0'
+      - 'lxc:asdf'
+      - '1'
+      - 'wordpress/3'
+      - 'asdf'
+  wordpress:
+    charm: 'cs:precise/wordpress-1'
+    num_units: 1
+machines:
+  3:
+    constraints: ''

=== modified file 'deployer/tests/test_deployment.py'
--- deployer/tests/test_deployment.py	2015-03-17 16:24:27 +0000
+++ deployer/tests/test_deployment.py	2015-03-24 17:39:12 +0000
@@ -8,14 +8,34 @@
 from .base import Base, skip_if_offline
 
 
+class FauxService(object):
+    """A fake service with a unit_placement attribute, used for testing
+    the sort functionality.
+    """
+
+    def __init__(self, name=None, unit_placement=None):
+        self.name = name
+        self.unit_placement = unit_placement
+
+
 class DeploymentTest(Base):
 
     def setUp(self):
         self.output = setup_logging(
             debug=True, verbose=True, stream=StringIO.StringIO())
 
-    def get_named_deployment_and_fetch(self, file_name, stack_name):
-        deployment = self.get_named_deployment(file_name, stack_name)
+    def get_named_deployment_and_fetch_v3(self, file_name, stack_name):
+        deployment = self.get_named_deployment_v3(file_name, stack_name)
+        # Fetch charms in order to allow proper late binding config and
+        # placement validation.
+        repo_path = self.mkdir()
+        os.mkdir(os.path.join(repo_path, "precise"))
+        deployment.repo_path = repo_path
+        deployment.fetch_charms()
+        return deployment
+
+    def get_deployment_and_fetch_v4(self, file_name):
+        deployment = self.get_deployment_v4(file_name)
         # Fetch charms in order to allow proper late binding config and
         # placement validation.
         repo_path = self.mkdir()
@@ -26,7 +46,7 @@
 
     @skip_if_offline
     def test_deployer(self):
-        d = self.get_named_deployment_and_fetch('blog.yaml', 'wordpress-prod')
+        d = self.get_named_deployment_and_fetch_v3('blog.yaml', 'wordpress-prod')
         services = d.get_services()
         self.assertTrue([s for s in services if s.name == "newrelic"])
 
@@ -53,7 +73,7 @@
             [('blog', 'db'), ('blog', 'cache'), ('blog', 'haproxy')])
 
     def test_maas_name_and_zone_placement(self):
-        d = self.get_named_deployment("stack-placement-maas.yml", "stack")
+        d = self.get_named_deployment_v3("stack-placement-maas.yml", "stack")
         d.validate_placement()
         placement = d.get_unit_placement('ceph', {})
         self.assertEqual(placement.get(0), "arnolt")
@@ -62,7 +82,7 @@
 
     @skip_if_offline
     def test_validate_placement_sorting(self):
-        d = self.get_named_deployment_and_fetch("stack-placement.yaml", "stack")
+        d = self.get_named_deployment_and_fetch_v3("stack-placement.yaml", "stack")
         services = d.get_services()
         self.assertEqual(services[0].name, 'nova-compute')
         try:
@@ -70,9 +90,76 @@
         except ErrorExit:
             self.fail("Should not fail")
 
+    def test_machines_placement_sort(self):
+        d = Deployment('test', None, None)
+        self.assertEqual(
+            d._machines_placement_sort(
+                FauxService(unit_placement=1),
+                FauxService()
+            ), 1)
+        self.assertEqual(
+            d._machines_placement_sort(
+                FauxService(),
+                FauxService(unit_placement=1)
+            ), -1)
+        self.assertEqual(
+            d._machines_placement_sort(
+                FauxService(name="x", unit_placement=['asdf']),
+                FauxService(name="y", unit_placement=['lxc:x/1'])
+            ), 1)
+        self.assertEqual(
+            d._machines_placement_sort(
+                FauxService(name="y", unit_placement=['lxc:x/1']),
+                FauxService(name="x", unit_placement=['asdf'])
+            ), -1)
+        self.assertEqual(
+            d._machines_placement_sort(
+                FauxService(name="x", unit_placement=['asdf']),
+                FauxService(name="y", unit_placement=['hjkl'])
+            ), -1)
+        self.assertEqual(
+            d._machines_placement_sort(
+                FauxService(name="x"),
+                FauxService(name="y")
+            ), -1)
+
+    def test_colocate(self):
+        status = {
+            'services': {
+                'foo': {
+                    'units': {
+                        '1': {
+                            'machine': 1
+                        },
+                        '2': {}
+                    }
+                }
+            }
+        }
+        d = self.get_named_deployment_v3("stack-placement.yaml", "stack")
+        p = d.get_unit_placement('ceph', status)
+        svc = FauxService(name='bar')
+
+        self.assertEqual(p.colocate(status, 'asdf', '1', '', svc),
+                         None)
+        self.assertIn('Service bar to be deployed with non-existent service '
+                      'asdf',
+                      self.output.getvalue())
+        self.assertEqual(p.colocate(status, 'foo', '2', '', svc),
+                         None)
+        self.assertIn('Service:bar, Deploy-with-service:foo, '
+                      'Requested-unit-index=2, Cannot solve, '
+                      'falling back to default placement',
+                      self.output.getvalue())
+        self.assertEqual(p.colocate(status, 'foo', '1', '', svc),
+                         None)
+        self.assertIn('Service:bar deploy-with unit missing machine 2',
+                      self.output.getvalue())
+        self.assertEqual(p.colocate(status, 'foo', '0', '', svc), 1)
+
     @skip_if_offline
     def test_validate_invalid_placement_nested(self):
-        d = self.get_named_deployment_and_fetch("stack-placement-invalid.yaml", "stack")
+        d = self.get_named_deployment_and_fetch_v3("stack-placement-invalid.yaml", "stack")
         services = d.get_services()
         self.assertEqual(services[0].name, 'nova-compute')
         try:
@@ -84,7 +171,7 @@
 
     @skip_if_offline
     def test_validate_invalid_placement_no_with_service(self):
-        d = self.get_named_deployment_and_fetch(
+        d = self.get_named_deployment_and_fetch_v3(
             "stack-placement-invalid-2.yaml", "stack")
         services = d.get_services()
         self.assertEqual(services[0].name, 'nova-compute')
@@ -96,9 +183,9 @@
             self.fail("Should fail")
 
     @skip_if_offline
-    def test_validate_invalid_placement_subordinate(self):
+    def test_validate_invalid_placement_subordinate_v3(self):
         # Placement validation fails if a subordinate charm is provided.
-        deployment = self.get_named_deployment_and_fetch(
+        deployment = self.get_named_deployment_and_fetch_v3(
             'stack-placement-invalid-subordinate.yaml', 'stack')
         with self.assertRaises(ErrorExit):
             deployment.validate_placement()
@@ -108,9 +195,21 @@
         self.assertIn(
             'Cannot place to a subordinate service: nova-compute -> nrpe\n',
             output)
+        
+    @skip_if_offline
+    def test_validate_invalid_placement_subordinate_v4(self):
+        # Placement validation fails if a subordinate charm is provided.
+        deployment = self.get_deployment_and_fetch_v4(
+                'placement-invalid-subordinate.yaml')
+        with self.assertRaises(ErrorExit):
+            deployment.validate_placement()
+        output = self.output.getvalue()
+        self.assertIn(
+            'Cannot place to a subordinate service: nova-compute -> nrpe\n',
+            output)
 
-    def test_get_unit_placement(self):
-        d = self.get_named_deployment("stack-placement.yaml", "stack")
+    def test_get_unit_placement_v3(self):
+        d = self.get_named_deployment_v3("stack-placement.yaml", "stack")
         status = {
             'services': {
                 'nova-compute': {
@@ -144,6 +243,195 @@
         self.assertEqual(placement.get(3), 'lxc:1')
         self.assertEqual(placement.get(4), 'lxc:3')
 
+    def test_fill_placement_v4(self):
+        d = self.get_deployment_v4('fill_placement.yaml')
+        self.assertEqual(
+            d.get_unit_placement('mediawiki1', 0).service.svc_data['to'],
+            ['new', 'new'])
+        self.assertEqual(
+            d.get_unit_placement('mediawiki2', 0).service.svc_data['to'],
+            ['0', '0'])
+        self.assertEqual(
+            d.get_unit_placement('mediawiki3', 0).service.svc_data['to'],
+            ['mediawiki1/0', 'mediawiki1/1'])
+
+    def test_parse_placement_v4(self):
+        # Short-cut to winding up with a valid placement.
+        d = self.get_deployment_v4('simple.yaml')
+        placement = d.get_unit_placement('mysql', {})
+
+        c, p, u = placement._parse_placement('mysql')
+        self.assertEqual(c, None)
+        self.assertEqual(p, 'mysql')
+        self.assertEqual(u, None)
+
+        c, p, u = placement._parse_placement('mysql/1')
+        self.assertEqual(c, None)
+        self.assertEqual(p, 'mysql')
+        self.assertEqual(u, '1')
+
+        c, p, u = placement._parse_placement('lxc:mysql')
+        self.assertEqual(c, 'lxc')
+        self.assertEqual(p, 'mysql')
+        self.assertEqual(u, None)
+
+    def test_validate_v4(self):
+        d = self.get_deployment_v4('validate.yaml')
+        placement = d.get_unit_placement('mysql', {})
+        feedback = placement.validate()
+        self.assertEqual(feedback.get_errors(), [
+            'Invalid container type: asdf service: mysql placement: asdf:0',
+            'Service placement to machine not supported mysql to asdf:0',
+            'Invalid target for container: lxc:asdf',
+            'Invalid service placement mysql to lxc:asdf',
+            'Service placement to machine not supported mysql to 1',
+            'Service unit does not exist, mysql to wordpress/3',
+            'Invalid service placement mysql to asdf'])
+
+    def test_get_unit_placement_v4_simple(self):
+        d = self.get_deployment_v4('simple.yaml')
+        placement = d.get_unit_placement('mysql', {})
+        self.assertEqual(placement.get(0), None)
+
+        placement = d.get_unit_placement('mediawiki', {})
+        self.assertEqual(placement.get(0), None)
+
+    def test_get_unit_placement_v4_placement(self):
+        d = self.get_deployment_v4('placement.yaml')
+        machines = {
+            '1': 1,
+            '2': 2,
+        }
+
+        d.set_machines(machines)
+
+        placement = d.get_unit_placement('mysql', {})
+        d.set_machines(machines)
+        self.assertEqual(placement.get(0), 2)
+
+        placement = d.get_unit_placement('mediawiki', {})
+        self.assertEqual(placement.get(0), 1)
+
+    def test_get_unit_placement_v4_hulk_smash(self):
+        d = self.get_deployment_v4('hulk-smash.yaml')
+        machines = {
+            '1': 1,
+        }
+        status = {
+            'services': {
+                'mediawiki': {
+                    'units': {
+                        'mediawiki/1': {'machine': 1}
+                    }
+                }
+            }
+        }
+
+        d.set_machines(machines)
+
+        placement = d.get_unit_placement('mysql', status)
+        self.assertEqual(placement.get(0), 1)
+
+        placement = d.get_unit_placement('mediawiki', status)
+        self.assertEqual(placement.get(0), 1)
+
+    def test_get_unit_placement_v4_hulk_smash_nounits(self):
+        d = self.get_deployment_v4('hulk-smash-nounits.yaml')
+        machines = {
+            '1': 1,
+        }
+        status = {
+            'services': {
+                'mediawiki': {
+                    'units': {
+                        'mediawiki/1': {'machine': 1}
+                    }
+                }
+            }
+        }
+
+        d.set_machines(machines)
+
+        placement = d.get_unit_placement('mysql', status)
+        self.assertEqual(placement.get(0), 1)
+
+        placement = d.get_unit_placement('mediawiki', status)
+        self.assertEqual(placement.get(0), 1)
+
+    def test_get_unit_placement_v4_hulk_smash_nounits_nomachines(self):
+        d = self.get_deployment_v4('hulk-smash-nounits-nomachines.yaml')
+        machines = {
+            '1': 1,
+        }
+        status = {
+            'services': {
+                'mediawiki': {
+                    'units': {
+                        'mediawiki/1': {'machine': 1}
+                    }
+                }
+            }
+        }
+
+        d.set_machines(machines)
+
+        placement = d.get_unit_placement('mysql', status)
+        self.assertEqual(placement.get(0), 1)
+
+        # Since we don't have a placement, even with the status, this should
+        # still be None.
+        placement = d.get_unit_placement('mediawiki', status)
+        self.assertEqual(placement.get(0), None)
+
+    def test_get_unit_placement_v4_container(self):
+        d = self.get_deployment_v4('container.yaml')
+        machines = {
+            '1': 1,
+        }
+        status = {
+            'services': {
+                'mediawiki': {
+                    'units': {
+                        'mediawiki/1': {'machine': 1},
+                    }
+                }
+            }
+        }
+
+        d.set_machines(machines)
+
+        placement = d.get_unit_placement('mysql', status)
+        self.assertEqual(placement.get(0), 'lxc:1')
+
+        placement = d.get_unit_placement('mediawiki', status)
+        self.assertEqual(placement.get(0), 1)
+
+    def test_get_unit_placement_v4_container_new(self):
+        d = self.get_deployment_v4('container-new.yaml')
+        machines = {
+            '1': 1,
+            'mysql/0': 2
+        }
+        status = {
+            'services': {
+                'mediawiki': {
+                    'units': {
+                        'mediawiki/1': {'machine': 1}
+                    }
+                }
+            }
+        }
+
+        d.set_machines(machines)
+
+        placement = d.get_unit_placement('mysql', status)
+        self.assertEqual(placement.get_new_machines_for_containers(),
+            ['mysql/0'])
+        self.assertEqual(placement.get(0), 'lxc:2')
+
+        placement = d.get_unit_placement('mediawiki', status)
+        self.assertEqual(placement.get(0), 1)
+
     def test_multiple_relations_no_weight(self):
         data = {"relations": {"wordpress": {"consumes": ["mysql"]},
                               "nginx": {"consumes": ["wordpress"]}}}
@@ -176,7 +464,7 @@
 
     def test_getting_service_names(self):
         # It is possible to retrieve the service names.
-        deployment = self.get_named_deployment("stack-placement.yaml", "stack")
+        deployment = self.get_named_deployment_v3("stack-placement.yaml", "stack")
         service_names = deployment.get_service_names()
         expected_service_names = [
             'ceph', 'mysql', 'nova-compute', 'quantum', 'semper', 'verity', 'lxc-service']
@@ -184,14 +472,14 @@
 
     def test_resolve_config_handles_empty_options(self):
         """resolve_config should handle options being "empty" lp:1361883"""
-        deployment = self.get_named_deployment("negative.cfg", "negative")
+        deployment = self.get_named_deployment_v3("negative.cfg", "negative")
         self.assertEqual(
             deployment.data["services"]["foo"]["options"], {})
         deployment.resolve_config()
 
     def test_resolve_config_handles_none_options(self):
         """resolve_config should handle options being "none" lp:1361883"""
-        deployment = self.get_named_deployment("negative.yaml", "negative")
+        deployment = self.get_named_deployment_v3("negative.yaml", "negative")
         self.assertEqual(
             deployment.data["services"]["foo"]["options"], None)
         deployment.resolve_config()

=== modified file 'deployer/tests/test_goenv.py'
--- deployer/tests/test_goenv.py	2014-09-03 13:17:33 +0000
+++ deployer/tests/test_goenv.py	2015-03-24 17:39:12 +0000
@@ -22,9 +22,10 @@
         self.env = GoEnvironment(
             os.environ.get("JUJU_ENV"), endpoint=self.endpoint)
         self.env.connect()
-        self.assertFalse(self.env.status().get('services'))
+        status = self.env.status()
+        self.assertFalse(status.get('services'))
         # Destroy everything.. consistent baseline
-        self.env.reset(terminate_machines=True, terminate_delay=240)
+        self.env.reset(terminate_machines=len(status['machines'].keys()) > 1, terminate_delay=240)
 
     def tearDown(self):
         self.env.reset(terminate_machines=True, terminate_delay=240)
@@ -37,7 +38,7 @@
         self.env.add_relation("test-db", "test-blog")
         self.env.add_units('test-blog', 1)
 
-        # Sleep cause juju core watches are eventually consistent (5s window)
+        # Sleep because juju core watches are eventually consistent (5s window)
         # and status rpc is broken (http://pad.lv/1203105)
         time.sleep(6)
         self.env.wait_for_units(timeout=800)
@@ -49,4 +50,26 @@
             services)
         for s in services:
             for k, u in status['services'][s]['units'].items():
-                self.assertEqual(u['agent-state'], "started")
+                self.assertIn(u['agent-state'], ("allocating", "started"))
+
+    def test_add_machine(self):
+        machine_name = self.env.add_machine()
+        
+        # Sleep because juju core watches are eventually consistent (5s window)
+        # and status rpc is broken (http://pad.lv/1203105)
+        time.sleep(6)
+        status = self.env.status()
+
+        self.assertIn(machine_name, status['machines'])
+
+    def test_set_annotation(self):
+        machine_name = self.env.add_machine()
+        self.env.set_annotation(machine_name, {'foo': 'bar'}, entity_type='machine')
+        
+        # Sleep because juju core watches are eventually consistent (5s window)
+        # and status rpc is broken (http://pad.lv/1203105)
+        time.sleep(6)
+        status = self.env.status()
+
+        self.assertIn('foo', self.env.client.get_annotation(
+            machine_name, 'machine')['Annotations'])

=== modified file 'deployer/tests/test_importer.py'
--- deployer/tests/test_importer.py	2015-03-17 10:40:34 +0000
+++ deployer/tests/test_importer.py	2015-03-24 17:39:12 +0000
@@ -75,3 +75,22 @@
         importer.run()
 
         self.assertFalse(env.add_relation.called, False)
+
+    @skip_if_offline
+    @mock.patch('deployer.action.importer.time')
+    def test_importer_add_machine_series(self, mock_time):
+        self.options.configs = [
+            os.path.join(self.test_data_dir, 'v4', 'series.yaml')]
+        stack = ConfigStack(self.options.configs)
+        deploy = stack.get(self.options.configs[0])
+        env = mock.MagicMock()
+        importer = Importer(env, deploy, self.options)
+        importer.run()
+
+        self.assertEqual(env.add_machine.call_count, 2)
+        self.assertEqual(
+            env.add_machine.call_args_list[0][1],
+            {'series': 'precise', 'constraints': 'mem=512M'})
+        self.assertEqual(
+            env.add_machine.call_args_list[1][1],
+            {'series': 'trusty', 'constraints': 'mem=512M'})

=== modified file 'deployer/utils.py'
--- deployer/utils.py	2014-09-30 19:26:26 +0000
+++ deployer/utils.py	2015-03-24 17:39:12 +0000
@@ -373,3 +373,22 @@
         if not 'default' in conf:
             raise ValueError("No Environment specified")
         return conf['default']
+
+def x_in_y(x, y):
+    """Check to see if the second argument is named in the first
+    argument's unit placement spec.
+    
+    Both arguments provided are services with unit placement directives.
+    If the first service appears in the second service's unit placement,
+    either colocated on a default unit, colocated with a specific unit,
+    or containerized alongside that service, then True is returned, False
+    otherwise.
+    """
+    for placement in y.unit_placement:
+        if ':' in placement:
+            _, placement = placement.split(':')
+        if '/' in placement:
+            placement, _ = placement.split('/')
+        if x.name == placement:
+            return True
+    return False