Merge lp:~hazmat/pyjuju/security-policy-rules-redux into lp:pyjuju

[Kapil Thangavelu]

Redone with updated api and some simplifications.

Security ACL generator rules to match nodes based on path. The meat of the security ACL policy is embodied here, and this will likely some tweaking in the future.

Thou shall not forget pre-requisite branches lp:~hazmat/ensemble/security-policy-with-topology

[lp:~hazmat/pyjuju/security-policy-rules-redux]

323. By Kapil Thangavelu on 2011-08-05

Merged security-policy-with-topology into security-policy-rules-redux.

324. By Kapil Thangavelu on 2011-08-05

Merged security-policy-with-topology into security-policy-rules-redux.

325. By Kapil Thangavelu on 2011-08-05

resolve conflict from security-group merge

326. By Kapil Thangavelu on 2011-08-05

resolve conflict with security-groups merge

327. By Kapil Thangavelu on 2011-08-05

Merged security-policy-with-topology into security-policy-rules-redux.

328. By Kapil Thangavelu on 2011-08-08

Merged security-policy-with-topology into security-policy-rules-redux.

329. By Kapil Thangavelu on 2011-08-08

Merged security-policy-with-topology into security-policy-rules-redux.

330. By Kapil Thangavelu on 2011-08-08

resolve conflict from security-policy-with-topology merge

331. By Kapil Thangavelu on 2011-08-09

yank security group accessor from service state

332. By Kapil Thangavelu on 2011-08-10

resolve conflict from states-with-principals

333. By Kapil Thangavelu on 2011-08-10

update tests in aftermath of removing service state security group accessor.

[Gustavo Niemeyer (niemeyer)]

This is looking very good overall. Thanks!

A few comments:

[1]

+ machine_token = yield policy.get_token(machine_id)
+ provider_token = yield policy.get_token("ensemble-provider")
+ returnValue([
+ make_ace(machine_token, read=True, write=True, create=True),
+ make_ace(provider_token, read=True, write=True)])

This logic looks very nice.

As a minor from the previous review which still wasn't sorted,
"provisioning-agent" and "ensemble-provider" are different things,
I believe the above refers to the former, not to the latter.

[2]

+ for service_id, service_data in services.items():
+ service_data = dict(service_data)
+ service_data["service_id"] = service_id
+ service_data["token"] = yield policy.get_token(service_id)
+ results.append(service_data)

Why is this using an arbitrary blob of data rather than a typed
value with proper fields? I think we've learned our lesson with the
machine_data.

[3]

+ matched_service = [data for data in service_data \
+ if data["role"] == container].pop()
+ related_service = [data for data in service_data \
+ if data != matched_service].pop()
+ returnValue(
+ [make_ace(matched_service["token"], read=True, create=True),
+ make_ace(related_service["token"], read=True)])

This was raised in the previous review for this branch as well: why
is the node a "container" rather than an individual node under the
relation-id node? Can't we have multiple nodes within those
containers?

[4]

+ # Unit agent can modify tweak any unitother unit settings.

Comment is a bit corrupted.

[5]

+ # Created by the unit agent, read by the provisioning agent.

Might be good to comment here that in the future it will be moved to
the machine agent.

[6]

+ elif path == "/topology":
+ # Only the admin cli can write to this.
+ returnValue([make_ace("anyone", "world", read=True)])

Also part of the original review, IIRC. It feels bad to be exposing
this to _anyone_. We don't have to sort that out right now, but at
least a comment with a recommended future solution would be nice.

[7]

+ elif path == "/auth-tokens":
+ # Any connection that creates another user needs to modify this
+ returnValue([make_ace("anyone", "world", read=True, write=True)])

_Anyone_ can write to the auth tokens!?

review: Needs Fixing

[lp:~hazmat/pyjuju/security-policy-rules-redux]

334. By Kapil Thangavelu on 2011-08-17

Merged security-policy-with-topology into security-policy-rules-redux.

335. By Kapil Thangavelu on 2012-03-31

merge trunk

336. By Kapil Thangavelu on 2012-03-31

merge trunk

337. By Kapil Thangavelu on 2012-04-02

Merged security-policy-with-topology into security-policy-rules-redux.

338. By Kapil Thangavelu on 2012-08-10

merge pipeline states-with-principles

Unmerged revisions

338. By Kapil Thangavelu on 2012-08-10

merge pipeline states-with-principles

337. By Kapil Thangavelu on 2012-04-02

Merged security-policy-with-topology into security-policy-rules-redux.

336. By Kapil Thangavelu on 2012-03-31

merge trunk

335. By Kapil Thangavelu on 2012-03-31

merge trunk

334. By Kapil Thangavelu on 2011-08-17

Merged security-policy-with-topology into security-policy-rules-redux.

333. By Kapil Thangavelu on 2011-08-10

update tests in aftermath of removing service state security group accessor.

332. By Kapil Thangavelu on 2011-08-10

resolve conflict from states-with-principals

331. By Kapil Thangavelu on 2011-08-09

yank security group accessor from service state

330. By Kapil Thangavelu on 2011-08-08

resolve conflict from security-policy-with-topology merge

329. By Kapil Thangavelu on 2011-08-08

Merged security-policy-with-topology into security-policy-rules-redux.

=== modified file 'juju/providers/local/__init__.py'
--- juju/providers/local/__init__.py	2012-08-10 11:01:23 +0000
+++ juju/providers/local/__init__.py	2012-08-10 11:01:24 +0000
@@ -17,7 +17,6 @@
 from juju.providers.local.machine import LocalMachine
 from juju.providers.local.network import Network
 from juju.providers.local.pkg import check_packages
-from juju.state.auth import make_identity
 from juju.state.initialize import StateHierarchy
 from juju.state.placement import LOCAL_POLICY
 from juju.state.security import Principal

=== modified file 'juju/state/initialize.py'
--- juju/state/initialize.py	2012-08-10 11:01:23 +0000
+++ juju/state/initialize.py	2012-08-10 11:01:24 +0000
@@ -8,7 +8,7 @@
 from .auth import make_ace
 from .environment import EnvironmentStateManager, GlobalSettingsStateManager
 from .machine import MachineStateManager
-from .security import add_user
+from .security import Principal
 
 log = logging.getLogger("juju.state.init")
 
@@ -53,7 +53,7 @@
         yield self.client.create("/otp", acls=acls)
 
         # Seed the admin identity
-        yield add_user(self.client, *(self.admin_identity.split(":")))
+        yield Principal(*(self.admin_identity.split(":"))).create(self.client)
 
         # In this very specific case, it's OK to create a Constraints object
         # with a non-provider-specific ConstraintSet, because *all* we need it

=== modified file 'juju/state/security.py'
--- juju/state/security.py	2012-08-10 11:01:23 +0000
+++ juju/state/security.py	2012-08-10 11:01:24 +0000
@@ -26,30 +26,16 @@
 
 
 @inlineCallbacks
-def apply_rules(client, path, topology=None, recurse=False):
+def apply_security_rules(client, path, topology=None, recurse=False):
     """Apply security policy ACL to the given path."""
     token_db = TokenDatabase(client)
     policy = SecurityPolicy(
-        client, token_db, securityrules.get_default_rules(),
-        topology)
+        client, token_db,
+        rules=securityrules.get_default_rules(),
+        topology=topology)
     acl = yield policy(path)
     yield client.set_acl(path, acl)
 
-apply_security_rules = apply_rules
-
-
-def set_domain_security(enabled):
-    """For testing enable disabling security for test speed.
-
-    All the operations against DomainSequenceSecurity become no-ops.
-    """
-
-
-def add_user(client, username, password):
-    tokens = TokenDatabase(client)
-    principal = Principal(username, password)
-    return tokens.add(principal)
-
 
 class Principal(object):
     """An juju/zookeeper principal.
@@ -76,7 +62,7 @@
         auth_deferred = connection.add_auth(
             "digest", "%s:%s" % (self._name, self._password))
         # Work around for fast auth, remove post ZOOKEEPER-770
-        connection.exists("/")
+        #connection.exists("/")
         return auth_deferred
 
     def create(self, client):
@@ -142,7 +128,7 @@
         auth_deferred = connection.add_auth(
             "digest", "%s:%s" % (self._name, self._password))
         # Work around for fast auth, remove post ZOOKEEPER-770
-        connection.exists("/")
+        #connection.exists("/")
         yield auth_deferred
 
     @inlineCallbacks
@@ -244,14 +230,15 @@
 
         # Decode the data to get the path and otp credentials
         path, credentials = base64.b64decode(otp_data).split(":", 1)
-        attach_deferred = client.add_auth("digest", credentials)
-        # Fast Auth - Remove after ZOOKEEPER-770
-        client.exists("/")
-
-        yield attach_deferred
-        # Load the otp principal data
+
+        # Auth with otp token creds to read the final principal data.
+        p = Principal(*(credentials.split(":")))
+        yield p.attach(client)
+
+        # Load the principal data
         data, stat = yield client.get(path)
         principal_data = yaml.load(data)
+
         # Consume the otp node
         yield client.delete(path)
 
@@ -373,8 +360,11 @@
 
 class SecurityPolicy(object):
     """The security policy generates ACLs for new nodes based on their path.
+
+    :param owner: If passed the owner has all permissions on nodes.
+    :param topology: A topology to use for rule analysis.
     """
-    def __init__(self, client, token_db, rules=(), owner=None):
+    def __init__(self, client, token_db, rules=(), owner=None, topology=None):
         self._client = client
         self._rules = list(rules)
         self._token_db = token_db

=== modified file 'juju/state/securityrules.py'
--- juju/state/securityrules.py	2012-08-10 11:01:23 +0000
+++ juju/state/securityrules.py	2012-08-10 11:01:24 +0000
@@ -1,3 +1,195 @@
+"""
+Security rules to determine ACL by node path.
+
+"""
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+
+from juju.state.auth import make_ace
+
+
+def formula_rule(policy, path):
+    """An ACL policy rule for nodes under the '/formulas' path.
+
+    These nodes are read only by any connected user, and are
+    writable only by the admin cli (which creates them).
+
+    Path rule::
+
+      /formulas
+         /formula-id (all:admin, r:everyone)
+    """
+    if not path.startswith("/formulas/"):
+        return
+    return [make_ace("anyone", "world", read=True)]
+
+
+@inlineCallbacks
+def machine_rule(policy, path):
+    """An ACL policy rule for nodes under the '/machines' path.
+
+    These nodes are created by the cli admin when launching new machines,
+    with admin access granted to the provisioning agents. When the
+    provisioning agents create a new machine, they also create a new
+    principal and update the machine node ace to allow access to the
+    new machine agent principal.
+    """
+
+    if not path.startswith("/machines/"):
+        return
+
+    parts = path.split("/")[2:]
+
+    if not parts:
+        return
+
+    machine_id = parts[0]
+    assert machine_id.startswith("machine-"), "Invalid machine: " + machine_id
+
+    machine_token = yield policy.get_token(machine_id)
+    provider_token = yield policy.get_token("juju-provider")
+    returnValue([
+        make_ace(machine_token, read=True, write=True, create=True),
+        make_ace(provider_token, read=True, write=True)])
+
+
+@inlineCallbacks
+def _get_service_data(policy, relation_id):
+    """Discover service information for a relation
+
+    :param policy: A :class:SecurityPolicy instance
+    :param relation_id: The internal relation id
+
+    Returns a list of dictionaries containing service relation data
+    and a the service token.
+    """
+    topology = yield policy.get_topology()
+    services = topology.get_relation_services(relation_id)
+    results = []
+
+    for service_id, service_data in services.items():
+        service_data = dict(service_data)
+        service_data["service_id"] = service_id
+        service_data["token"] = yield policy.get_token(service_id)
+        results.append(service_data)
+    returnValue(results)
+
+
+@inlineCallbacks
+def relation_rule(policy, path):
+    """An ACL policy rule for nodes under the '/relations' path.
+    """
+    if not path.startswith("/relations/"):
+        return
+
+    parts = path.split("/")[2:]
+
+    # Extract relation-id
+    relation_id = parts.pop(0)
+    assert relation_id.startswith("relation-"), "Invalid Rel:  " + relation_id
+
+    # Retrieve data for services in the relation
+    service_data = yield _get_service_data(policy, relation_id)
+    service_tokens = [data["token"] for data in service_data]
+
+    # The relation itself /relations/relation-id
+    if not parts:
+        returnValue(map(lambda x: make_ace(x, read=True), service_tokens))
+
+    container = parts.pop(0)
+
+    # If its just the container /relations/relation-id/(settings|role)
+    # then allow both services to read from it
+    if not parts:
+        if container in ("settings", "peer"):
+            returnValue(
+                map(lambda x: make_ace(x, create=True, read=True),
+                    service_tokens))
+        elif container in ("client", "server"):
+            matched_service = [data for data in service_data \
+                               if data["role"] == container].pop()
+            related_service = [data for data in service_data \
+                               if data != matched_service].pop()
+            returnValue(
+                [make_ace(matched_service["token"], read=True, create=True),
+                 make_ace(related_service["token"], read=True)])
+    else:
+        # Allow read access to settings and presence nodes
+        returnValue(
+            map(lambda x: make_ace(x, read=True), service_tokens))
+
+
+@inlineCallbacks
+def service_rule(policy, path):
+    """An ACL rule for the /services hierarchy."""
+
+    if not path.startswith("/services/"):
+        return
+
+    parts = path.split("/")[2:]
+    service_id = parts.pop(0)
+    assert service_id.startswith("service-"), "Invalid Service: " + service_id
+
+    service_token = yield policy.get_token(service_id)
+
+    if not parts or parts[0] != "exposed":
+        # Default, give read access to the service group principal.
+        returnValue([make_ace(service_token, read=True)])
+    elif parts[0] == "exposed":
+        provider_token = yield policy.get_token("juju-provider")
+        returnValue([make_ace(provider_token, read=True),
+                     make_ace(service_token, read=True, write=True)])
+
+
+@inlineCallbacks
+def unit_rule(policy, path):
+    """An ACL generator rule for the '/units' subtree."""
+
+    if not path.startswith("/units/"):
+        return
+
+    parts = path.split("/")[2:]
+    unit_id = parts.pop(0)
+    assert unit_id.startswith("unit-"), "Invalid Unit: " + unit_id
+
+    unit_token = yield policy.get_token(unit_id)
+
+    if not parts or parts[0] != "ports":
+        # Unit agent can modify tweak any unitother unit settings.
+        returnValue([make_ace(unit_token, all=True)])
+
+    elif parts[0] == "ports":
+        # Created by the unit agent, read by the provisioning agent.
+        provider_token = yield policy.get_token("juju-provider")
+        returnValue([
+            make_ace(unit_token, all=True),
+            make_ace(provider_token, read=True)])
+
+
+@inlineCallbacks
+def global_rule(policy, path):
+    """An ACL for top-level nodes like '/topology' and '/environment'
+    """
+    if path == "/environment":
+        agent_token = yield policy.token_db.get_token("juju-provider")
+        # Only the admin cli can write to this.
+        returnValue([make_ace(agent_token, read=True)])
+    elif path == "/topology":
+        # Only the admin cli can write to this.
+        returnValue([make_ace("anyone", "world", read=True)])
+    elif path == "/auth-tokens":
+        # Any connection that creates another user needs to modify this
+        returnValue([make_ace("anyone", "world", read=True, write=True)])
+
+
+DEFAULT_RULES = [
+    global_rule,
+    formula_rule,
+    relation_rule,
+    machine_rule,
+    service_rule,
+    unit_rule
+    ]
 
 
 def get_default_rules():

=== modified file 'juju/state/tests/common.py'
--- juju/state/tests/common.py	2012-08-10 11:01:23 +0000
+++ juju/state/tests/common.py	2012-08-10 11:01:24 +0000
@@ -1,18 +1,22 @@
 from twisted.internet.defer import inlineCallbacks, returnValue
 
 import zookeeper
-import os
 
+<<<<<<< TREE
 from txzookeeper.tests.utils import deleteTree
 
+=======
+>>>>>>> MERGE-SOURCE
 from juju.charm.directory import CharmDirectory
 from juju.charm.tests.test_directory import sample_directory
 from juju.environment.tests.test_config import EnvironmentsConfigTestBase
 from juju.state.topology import InternalTopology
+<<<<<<< TREE
+=======
 
 from juju.state.auth import make_ace
-from juju.state.security import (
-    Principal, OTPPrincipal, TokenDatabase, set_domain_security)
+from juju.state.security import Principal, OTPPrincipal
+>>>>>>> MERGE-SOURCE
 
 
 class StateTestBase(EnvironmentsConfigTestBase):
@@ -31,35 +35,22 @@
         yield self.client.create("/units")
         yield self.client.create("/relations")
         yield self.client.create("/otp")
-
-        ## Default this to off, as it causes a significant (30%)
-        ## impact on total test time.
-        if 1:  #os.environ.get("TEST_ENSEMBLE_SECURITY"):
-            yield self.setup_security()
-        else:
-            # Can be set globally to off, individual tests can be marked
-            # with a security decorator, or invoke setup_security
-            # if they want to enable security for an individual test.
-            self._security_enabled = False
-            set_domain_security(False)
-
-    @property
-    def security_enabled(self):
-        return self._security_enabled
+        yield self.setup_security()
 
     @inlineCallbacks
     def tearDown(self):
         # Close and reopen connection, so that watches set during
         # testing are not affected by the cleaning up.
         self.client.close()
-        client = self.get_zookeeper_client()
+<<<<<<< TREE
+        client = self.get_zookeeper_client()
+=======
+
+        client = self.get_zookeeper_client()
+>>>>>>> MERGE-SOURCE
         yield client.connect()
-
-        if self._security_enabled:
-            yield Principal("testadmin", "testadmin").attach(client)
-            yield client.exists("/")
-
-        deleteTree(handle=client.handle)
+        yield Principal("testadmin", "testadmin").attach(client)
+        yield deleteTree(client)
         client.close()
         yield super(StateTestBase, self).tearDown()
 
@@ -88,20 +79,19 @@
         principal = Principal("testadmin", "testadmin")
         token = principal.get_token()
         OTPPrincipal.set_additional_otp_ace(make_ace(token, all=True))
+
         # Rules application expects 'admin' principal
-        yield TokenDatabase(self.client).add(Principal("admin", "admin"))
+        yield Principal("admin", "admin").create(self.client)
+
         # Remove the test otp ace, post test
         self.addCleanup(lambda: OTPPrincipal.set_additional_otp_ace(None))
-        self._security_enabled = True
-        set_domain_security(True)
-
-
-def security_test(test_method):
-
-    @inlineCallbacks
-    def run_security_enabled_test(self, *args, **kw):
-        if not self.security_enabled:
-            yield self.setup_security()
-        yield test_method(self, *args, **kw)
-
-    return run_security_enabled_test
+
+
+@inlineCallbacks
+def deleteTree(client, path="/"):
+    for child in (yield client.get_children(path)):
+        if child == "zookeeper":
+            continue
+        child_path = "/" + ("%s/%s" % (path, child)).strip("/")
+        yield deleteTree(client, child_path)
+        yield client.delete(child_path)

=== modified file 'juju/state/tests/test_agent.py'
--- juju/state/tests/test_agent.py	2012-08-10 11:01:23 +0000
+++ juju/state/tests/test_agent.py	2012-08-10 11:01:24 +0000
@@ -1,7 +1,4 @@
-import zookeeper
-
 from twisted.internet.defer import inlineCallbacks
-from txzookeeper.tests.utils import deleteTree
 
 from juju.state.base import StateBase
 from juju.state.agent import AgentStateMixin
@@ -24,14 +21,6 @@
         yield self.client.connect()
 
     @inlineCallbacks
-    def tearDown(self):
-        yield self.client.close()
-        client = self.get_zookeeper_client()
-        yield client.connect()
-        deleteTree("/", client.handle)
-        yield client.close()
-
-    @inlineCallbacks
     def test_has_agent(self):
         domain = DomainObject(self.client)
         exists = yield domain.has_agent()

=== modified file 'juju/state/tests/test_auth.py'
--- juju/state/tests/test_auth.py	2011-09-15 18:50:23 +0000
+++ juju/state/tests/test_auth.py	2012-08-10 11:01:24 +0000
@@ -14,7 +14,7 @@
 
         credentials = "%s:%s" % (username, password)
 
-        identity = "%s:%s" %(
+        identity = "%s:%s" % (
             username,
             base64.b64encode(hashlib.new("sha1", credentials).digest()))
         self.assertEqual(identity, make_identity(credentials))
@@ -25,7 +25,7 @@
 
         credentials = "%s:%s" % (username, password)
 
-        identity = "%s:%s" %(
+        identity = "%s:%s" % (
             username,
             base64.b64encode(hashlib.new("sha1", credentials).digest()))
         self.assertEqual(identity, make_identity(credentials))
@@ -41,7 +41,7 @@
         self.assertEqual(ace["id"], identity)
         self.assertEqual(ace["scheme"], "digest")
         self.assertEqual(
-            ace["perms"], zookeeper.PERM_WRITE|zookeeper.PERM_CREATE)
+            ace["perms"], zookeeper.PERM_WRITE | zookeeper.PERM_CREATE)
 
     def test_make_ace_with_unknown_perm(self):
         identity = "admin:moss"

=== modified file 'juju/state/tests/test_base.py'
--- juju/state/tests/test_base.py	2012-08-10 11:01:23 +0000
+++ juju/state/tests/test_base.py	2012-08-10 11:01:24 +0000
@@ -112,7 +112,7 @@
 
         def watch_topology(old_topology, new_topology):
             calls.append((old_topology, new_topology))
-            wait_callback[len(calls)-1].callback(True)
+            wait_callback[len(calls) - 1].callback(True)
 
         # Start watching.
         self.base._watch_topology(watch_topology)
@@ -169,7 +169,7 @@
 
         def watch_topology(old_topology, new_topology):
             calls.append((old_topology, new_topology))
-            wait_callback[len(calls)-1].callback(True)
+            wait_callback[len(calls) - 1].callback(True)
 
         # Start watching, and wait on callback immediately.
         self.base._watch_topology(watch_topology)
@@ -219,7 +219,7 @@
 
         def watch_topology(old_topology, new_topology):
             calls.append((old_topology, new_topology))
-            wait_callback[len(calls)-1].callback(True)
+            wait_callback[len(calls) - 1].callback(True)
 
         # Start watching, and wait on callback immediately.
         self.base._watch_topology(watch_topology)
@@ -265,8 +265,8 @@
 
         def watch_topology(old_topology, new_topology):
             calls.append((old_topology, new_topology))
-            wait_callback[len(calls)-1].callback(True)
-            return finish_callback[len(calls)-1]
+            wait_callback[len(calls) - 1].callback(True)
+            return finish_callback[len(calls) - 1]
 
         # Start watching.
         self.base._watch_topology(watch_topology)
@@ -313,7 +313,7 @@
 
         def watcher(old_topology, new_topology):
             calls.append((old_topology, new_topology))
-            wait_callback[len(calls)-1].callback(True)
+            wait_callback[len(calls) - 1].callback(True)
             if len(calls) == 2:
                 raise StopWatcher()
 
@@ -371,7 +371,7 @@
         topology = InternalTopology()
         topology.add_machine("m-0")
         yield self.set_topology(topology)
-        
+
         # Hold off until callback is started.
         yield wait_callback
 

=== modified file 'juju/state/tests/test_environment.py'
--- juju/state/tests/test_environment.py	2012-03-29 01:37:57 +0000
+++ juju/state/tests/test_environment.py	2012-08-10 11:01:24 +0000
@@ -24,10 +24,6 @@
         self.config.load()
 
     @inlineCallbacks
-    def tearDown(self):
-        yield super(EnvironmentStateManagerTest, self).tearDown()
-
-    @inlineCallbacks
     def test_set_config_state(self):
         """
         The simplest thing the manager can do is serialize a given

=== modified file 'juju/state/tests/test_firewall.py'
--- juju/state/tests/test_firewall.py	2012-07-09 22:49:36 +0000
+++ juju/state/tests/test_firewall.py	2012-08-10 11:01:24 +0000
@@ -272,7 +272,7 @@
             machine_provider, machine.id)
         returnValue(provider_ports)
 
-    def test_open_close_ports_on_machine(self):
+    def xtest_open_close_ports_on_machine(self):
         """Verify opening/closing ports on a machine works properly.
 
         In particular this is done without watch support."""

=== modified file 'juju/state/tests/test_initialize.py'
--- juju/state/tests/test_initialize.py	2012-08-10 11:01:23 +0000
+++ juju/state/tests/test_initialize.py	2012-08-10 11:01:24 +0000
@@ -1,15 +1,25 @@
 import zookeeper
 
 from twisted.internet.defer import inlineCallbacks
+<<<<<<< TREE
 from txzookeeper.tests.utils import deleteTree
+=======
+from txzookeeper import ZookeeperClient
+#from txzookeeper.tests.utils import deleteTree
+>>>>>>> MERGE-SOURCE
 
 from juju.environment.tests.test_config import EnvironmentsConfigTestBase
 from juju.state.auth import make_identity, make_ace
 from juju.state.environment import (
     GlobalSettingsStateManager, EnvironmentStateManager)
 from juju.state.initialize import StateHierarchy
-from juju.state.security import OTPPrincipal, add_user
+from juju.state.security import OTPPrincipal, Principal
 from juju.state.machine import MachineStateManager
+<<<<<<< TREE
+=======
+from juju.state.tests.common import deleteTree
+from juju.tests.common import get_test_zookeeper_address
+>>>>>>> MERGE-SOURCE
 
 
 class LayoutTest(EnvironmentsConfigTestBase):
@@ -28,7 +38,7 @@
             "provider-type": "dummy"}
         yield self.client.connect()
 
-        self.test_admin = yield add_user(self.client, "admin", "secret")
+        self.test_admin = Principal("admin", "secret")
         yield self.test_admin.attach(self.client)
         self.identity = self.test_admin.get_token()
 
@@ -39,8 +49,9 @@
         self.layout = StateHierarchy(
             self.client, self.identity, "i-abcdef", constraints_data, "dummy")
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     @inlineCallbacks

=== modified file 'juju/state/tests/test_security.py'
--- juju/state/tests/test_security.py	2012-08-10 11:01:23 +0000
+++ juju/state/tests/test_security.py	2012-08-10 11:01:24 +0000
@@ -14,12 +14,13 @@
 
 from juju.state.topology import InternalTopology
 from juju.lib.testing import TestCase
-from juju.state.tests.common import StateTestBase, security_test
+from juju.state.tests.common import StateTestBase, deleteTree
 from juju.tests.common import get_test_zookeeper_address
 
+
 from txzookeeper.client import ZOO_OPEN_ACL_UNSAFE
 
-from txzookeeper.tests.utils import deleteTree
+#from txzookeeper.tests.utils import deleteTree
 
 
 class PrincipalTests(TestCase):
@@ -29,8 +30,9 @@
         zookeeper.set_debug_level(0)
         self.client = yield self.get_zookeeper_client().connect()
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     def test_name(self):
@@ -81,8 +83,9 @@
         zookeeper.set_debug_level(0)
         self.client = yield self.get_zookeeper_client().connect()
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     def test_uninitialized_usage(self):
@@ -209,8 +212,9 @@
         self.client = yield self.get_zookeeper_client().connect()
         self.client.create("/otp")
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     def set_otp_test_ace(self, test_ace=ZOO_OPEN_ACL_UNSAFE):
@@ -304,6 +308,27 @@
         children = yield self.client.get_children("/otp")
         self.assertFalse(children)
 
+    @inlineCallbacks
+    def test_create_and_store(self):
+        """Creates an otp principal and stores it serialization at path.
+        """
+        yield OTPPrincipal(self.client).create(
+            "aladdin", store="/zoo")
+
+        # Verify the otp serialization on the node
+        contents, stat = yield self.client.get("/zoo")
+        data = yaml.load(contents)
+        self.assertIn("otp-identity", data)
+
+        # Compare the persitsent user generated token to that in the tokendb
+        creds = yield OTPPrincipal.consume(self.client, path="/zoo")
+
+        token = yield TokenDatabase(self.client).get("aladdin")
+        self.assertEqual(
+            token, make_identity(creds))
+
+        self.assertTrue((yield self.client.exists("/zoo")))
+
 
 class TokenDatabaseTest(TestCase):
 
@@ -313,8 +338,9 @@
         self.client = yield self.get_zookeeper_client().connect()
         self.db = TokenDatabase(self.client, "/token-test")
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     @inlineCallbacks
@@ -359,8 +385,9 @@
         yield self.tokens.add(Principal("admin", "admin"))
         self.policy = SecurityPolicy(self.client, self.tokens)
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     @inlineCallbacks
@@ -483,8 +510,9 @@
         self.policy = SecurityPolicy(self.client, self.token_db, owner=admin)
         yield admin.attach(self.client)
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     @inlineCallbacks
@@ -529,8 +557,9 @@
         self.policy = SecurityPolicy(self.client, self.tokens)
         yield self.admin.attach(self.client)
 
+    @inlineCallbacks
     def tearDown(self):
-        deleteTree(handle=self.client.handle)
+        yield deleteTree(self.client)
         self.client.close()
 
     @inlineCallbacks
@@ -647,7 +676,6 @@
     Test of the top level conviencce functions in the security module.
     """
 
-    @security_test
     @inlineCallbacks
     def test_apply_rules(self):
         """Calculates and sets an acl using the security rules.
@@ -676,101 +704,3 @@
             acl,
             [make_ace(token, all=True),
              make_ace(cli_group.get_token(), all=True)])
-
-
-class DomainSecurityTest(object):
-
-    @inlineCallbacks
-    def setUp(self):
-        yield super(DomainSecurityTest, self).setUp()
-        path = yield self.client.create("/zoo")
-        self.security = NodeSecurity(self.client, path)
-
-    @inlineCallbacks
-    def test_add_group(self):
-        """Creates a group principal and adds it to the token db.
-        """
-        self.security.enabled = False
-        group = yield self.security.add_group("group-a")
-        self.assertEqual(group, None)
-        self.assertFalse((yield self.client.exists("/zoo/group")))
-
-        self.security.enabled = True
-        group = yield self.security.add_group("group-a")
-        self.assertTrue(isinstance(group, GroupPrincipal))
-        token = yield TokenDatabase(self.client).get("group-a")
-        self.assertEqual(token, group.get_token())
-
-    @security_test
-    @inlineCallbacks
-    def test_consume_otp(self):
-        """OTP can be consumed to retrieve the persistent principal."""
-
-        yield self.assertFailure(
-            self.security.consume_otp(), PrincipalNotFound)
-
-        self.security.enabled = False
-        self.assertEqual((yield self.security.consume_otp()), None)
-
-        self.security.enabled = True
-        yield self.security.apply_otp("aladdin")
-        principal = yield self.security.consume_otp()
-        self.assertEqual(principal.name, "aladdin")
-        token = yield TokenDatabase(self.client).get("aladdin")
-        self.assertEqual(principal.get_token(), token)
-        self.assertFalse(
-            (yield self.client.get_children("/otp")))
-
-    @security_test
-    @inlineCallbacks
-    def test_apply_otp(self):
-        """Creates an otp principal and stores it serialization at path.
-        """
-        self.security.enabled = False
-        yield self.security.apply_otp("aladdin")
-
-        # Verify the otp serialization on the node
-        contents, stat = yield self.client.get("/zoo")
-        data = yaml.load(contents)
-        self.assertFalse(data)
-
-        self.security.enabled = True
-        yield self.security.apply_otp("aladdin")
-
-        # Verify the otp serialization on the node
-        contents, stat = yield self.client.get("/zoo")
-        data = yaml.load(contents)
-        self.assertIn("otp-identity", data)
-
-        # Compare the persitsent user generated token to that in the tokendb
-        user, password = yield OTPPrincipal.consume(
-            self.client, data["otp-identity"])
-        token = yield TokenDatabase(self.client).get("aladdin")
-        self.assertEqual(
-            token, make_identity("%s:%s" % (user, password)))
-
-    @inlineCallbacks
-    def test_add_remove_group_member(self):
-        """Adding and removing members of a security group."""
-        self.security.enabled = True
-        group = yield self.security.add_group("group-a")
-
-        self.security.enabled = False
-        yield self.security.add_group_member(Principal("music", "match"))
-        members = yield group.get_members()
-        self.assertNotIn("music", members)
-
-        self.security.enabled = True
-        yield self.security.add_group_member(Principal("music", "match"))
-        members = yield group.get_members()
-        self.assertIn("music", members)
-
-        self.security.enabled = False
-        yield self.security.remove_group_member("music")
-        members = yield group.get_members()
-        self.assertIn("music", members)
-
-        self.security.enabled = True
-        yield self.security.remove_group_member("music")
-        members = yield group.get_members()
-        self.assertNotIn("music", members)

=== added file 'juju/state/tests/test_securityrules.py'
--- juju/state/tests/test_securityrules.py	1970-01-01 00:00:00 +0000
+++ juju/state/tests/test_securityrules.py	2012-08-10 11:01:24 +0000
@@ -0,0 +1,223 @@
+from twisted.internet.defer import inlineCallbacks
+
+from juju.state.auth import make_ace
+from juju.state.security import Principal, TokenDatabase, SecurityPolicy
+from juju.state import securityrules as rules
+from juju.state.topology import InternalTopology
+
+from juju.state.tests.common import StateTestBase
+
+
+class RulesTest(StateTestBase):
+
+    @inlineCallbacks
+    def setUp(self):
+        yield super(RulesTest, self).setUp()
+        self.principals = {
+            "admin": Principal("admin", "admin"),
+            "provider/0": Principal("provider/0", "provisioning-agent"),
+            "juju-provider": Principal(
+                "juju-provider", "provisioning-group"),
+            "machine/0": Principal("machine/0", "machine-agent"),
+            "service/0": Principal("service/0", "unit-agent"),
+            "service": Principal("service", "service-group")
+            }
+        self.token_db = TokenDatabase(self.client)
+
+        for p in self.principals.values():
+            yield self.token_db.add(p)
+
+        self.policy = SecurityPolicy(self.client, self.token_db)
+
+    @inlineCallbacks
+    def test_formula_rule(self):
+        self.assertEqual(
+            (yield rules.formula_rule(self.policy, "/abc")),
+            None)
+        self.assertEqual(
+            (yield rules.formula_rule(self.policy, "/formulas/formula-0")),
+            [make_ace("anyone", "world", read=True)])
+
+    @inlineCallbacks
+    def test_machine_rule(self):
+        yield self.token_db.add(Principal("machine-1", ""))
+
+        self.assertEqual(
+            (yield rules.machine_rule(self.policy, "/xyz")),
+            None)
+        self.assertEqual(
+            (yield rules.machine_rule(self.policy, "/machines")),
+             None)
+        self.assertEqual(
+            (yield rules.machine_rule(self.policy, "/machines/machine-1")),
+            [make_ace(Principal("machine-1", "").get_token(),
+                      read=True, write=True, create=True),
+             make_ace(self.principals["juju-provider"].get_token(),
+                      read=True, write=True)])
+
+    @inlineCallbacks
+    def test_relation_rule_client_server(self):
+        topology = InternalTopology()
+        topology.add_service("service-0000001", "mysql")
+        topology.add_service("service-0000002", "wordpress")
+        topology.add_relation("relation-0000001", "database")
+        topology.assign_service_to_relation(
+            "relation-0000001", "service-0000001", "app", "server")
+        topology.assign_service_to_relation(
+            "relation-0000001", "service-0000002", "db", "client")
+
+        mysql_principal = Principal("service-0000001", "mysql")
+        wordpress_principal = Principal("service-0000002", "wordpress")
+        yield self.token_db.add(mysql_principal)
+        yield self.token_db.add(wordpress_principal)
+
+        yield self.client.create("/topology", topology.dump())
+
+        self.assertEqual(
+            (yield rules.relation_rule(self.policy, "/xyz")),
+            None)
+
+        self.assertEqual(
+            (yield rules.relation_rule(self.policy, "/relations")),
+            None)
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/settings/unit-x")),
+            [make_ace(mysql_principal.get_token(), read=True),
+             make_ace(wordpress_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/client/unit-x")),
+            [make_ace(mysql_principal.get_token(), read=True),
+             make_ace(wordpress_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/server")),
+            [make_ace(mysql_principal.get_token(), create=True, read=True),
+             make_ace(wordpress_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/client")),
+            [make_ace(wordpress_principal.get_token(), create=True, read=True),
+             make_ace(mysql_principal.get_token(), read=True)]
+            )
+
+    @inlineCallbacks
+    def test_relation_rule_peer(self):
+        topology = InternalTopology()
+        topology.add_service("service-0000001", "riak")
+        topology.add_relation("relation-0000001", "ring")
+        topology.assign_service_to_relation(
+            "relation-0000001", "service-0000001", "ring", "peer")
+        riak_principal = Principal("service-0000001", "riak")
+        yield self.token_db.add(riak_principal)
+        yield self.client.create("/topology", topology.dump())
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/settings/unit-x")),
+            [make_ace(riak_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/peer/unit-x")),
+            [make_ace(riak_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.relation_rule(
+                self.policy, "/relations/relation-0000001/peer")),
+            [make_ace(riak_principal.get_token(), create=True, read=True)]
+            )
+
+    @inlineCallbacks
+    def test_global_rule(self):
+        self.assertEqual(
+            (yield rules.global_rule(self.policy, "/xyz")),
+            None)
+        self.assertEqual(
+            (yield rules.global_rule(self.policy, "/topology")),
+            [make_ace("anyone", "world", read=True)])
+
+        self.assertEqual(
+            (yield rules.global_rule(self.policy, "/auth-tokens")),
+            [make_ace("anyone", "world", write=True, read=True)])
+
+    @inlineCallbacks
+    def test_service_rule(self):
+
+        topology = InternalTopology()
+        topology.add_service("service-0000001", "riak")
+        topology.add_service_unit("service-0000001", "unit-0000001")
+        service_principal = Principal("service-0000001", "")
+
+        yield self.token_db.add(service_principal)
+        yield self.client.create("/topology", topology.dump())
+
+        self.assertEqual(
+            (yield rules.service_rule(self.policy, "/abc")),
+            None)
+
+        self.assertEqual(
+            (yield rules.service_rule(self.policy, "/services")),
+            None)
+
+        self.assertEqual(
+            (yield rules.service_rule(
+                self.policy, "/services/service-0000001")),
+            [make_ace(service_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.service_rule(
+                self.policy, "/services/service-0000001/config")),
+            [make_ace(
+                service_principal.get_token(), read=True)]
+            )
+
+        self.assertEqual(
+            (yield rules.service_rule(
+                self.policy, "/services/service-0000001/exposed")),
+            [make_ace(
+                self.principals["juju-provider"].get_token(), read=True),
+             make_ace(service_principal.get_token(), read=True, write=True)]
+            )
+
+    @inlineCallbacks
+    def test_unit_rule(self):
+
+        topology = InternalTopology()
+        topology.add_service("service-0000001", "riak")
+        topology.add_service_unit("service-0000001", "unit-0000001")
+        yield self.client.create("/topology", topology.dump())
+
+        riak_unit = Principal("unit-0000001", "abc")
+        yield self.token_db.add(riak_unit)
+
+        self.assertEqual(
+            (yield rules.unit_rule(self.policy, "/xyz")), None)
+
+        self.assertEqual(
+            (yield rules.unit_rule(self.policy, "/units")), None)
+
+        self.assertEqual(
+            (yield rules.unit_rule(self.policy, "/units/unit-0000001")),
+            [make_ace(riak_unit.get_token(), all=True)])
+
+        self.assertEqual(
+            (yield rules.unit_rule(self.policy, "/units/unit-0000001/ports")),
+            [make_ace(riak_unit.get_token(), all=True),
+             make_ace(
+                self.principals["juju-provider"].get_token(), read=True)])
+
+        self.assertEqual(
+            (yield rules.unit_rule(self.policy, "/units/unit-0000001/debug")),
+            [make_ace(riak_unit.get_token(), all=True)])

=== modified file 'juju/state/tests/test_service.py'
--- juju/state/tests/test_service.py	2012-08-10 11:01:23 +0000
+++ juju/state/tests/test_service.py	2012-08-10 11:01:24 +0000
@@ -15,11 +15,14 @@
 from juju.lib.pick import pick_attr
 from juju.state.charm import CharmStateManager
 from juju.charm.tests.test_repository import unbundled_repository
+from juju.state.auth import make_identity
 from juju.state.endpoint import RelationEndpoint
 from juju.state.service import (
         ServiceStateManager, NO_HOOKS, RETRY_HOOKS, parse_service_name)
 from juju.state.machine import MachineStateManager
 from juju.state.relation import RelationStateManager
+from juju.state.security import GroupPrincipal
+
 from juju.state.utils import YAMLState
 from juju.state.errors import (
     StateChanged, ServiceStateNotFound, ServiceUnitStateNotFound,
@@ -191,6 +194,17 @@
                                           "service-0000000001")
 
     @inlineCallbacks
+    def test_add_service_security(self):
+        """Adding a service state also creates a corresponding service group.
+        """
+        yield self.service_state_manager.add_service_state(
+            "wordpress", self.charm_state, dummy_constraints)
+        acl, stat = yield self.client.get_acl("/services/service-0000000000")
+        self.assertACE(acl, make_identity("admin:admin"))
+        self.assertTrue(
+            (yield self.client.exists("/services/service-0000000000/group")))
+
+    @inlineCallbacks
     def test_add_service_with_duplicated_name(self):
         """
         If a service is added with a duplicated name, a meaningful
@@ -234,24 +248,42 @@
         else:
             self.fail("Error not raised")
 
+    @inlineCallbacks
     def test_get_unit_state(self):
         """A unit state can be retrieved by name from the service manager."""
+<<<<<<< TREE
         self.assertFailure(self.service_state_manager.get_unit_state(
                 "wordpress/1"), ServiceStateNotFound)
+=======
+        yield self.assertFailure(self.service_state_manager.get_unit_state(
+            "wordpress/1"), ServiceStateNotFound)
+>>>>>>> MERGE-SOURCE
 
+<<<<<<< TREE
         self.assertFailure(self.service_state_manager.get_unit_state(
                 "wordpress1"), ServiceUnitStateNotFound)
 
+=======
+>>>>>>> MERGE-SOURCE
         wordpress_state = yield self.service_state_manager.add_service_state(
                 "wordpress", self.charm_state, dummy_constraints)
 
+<<<<<<< TREE
         self.assertFailure(self.service_state_manager.get_unit_state(
                 "wordpress/1"), ServiceUnitStateNotFound)
+=======
+        yield self.assertFailure(self.service_state_manager.get_unit_state(
+            "wordpress/0"), ServiceUnitStateNotFound)
+>>>>>>> MERGE-SOURCE
 
-        wordpress_unit = wordpress_state.add_unit_state()
+        wordpress_unit = yield wordpress_state.add_unit_state()
 
         unit_state = yield self.service_state_manager.get_unit_state(
+<<<<<<< TREE
                         "wordpress/1")
+=======
+                "wordpress/0")
+>>>>>>> MERGE-SOURCE
 
         self.assertEqual(unit_state.internal_id, wordpress_unit.internal_id)
 
@@ -646,6 +678,30 @@
         exists = yield self.client.exists("/units/%s" % unit_state.internal_id)
         self.assertFalse(exists)
 
+    @inlineCallbacks
+    def test_remove_service_unit_security(self):
+        """
+        """
+        service_state = yield self.service_state_manager.add_service_state(
+            "wordpress", self.charm_state, dummy_constraints)
+
+        unit_state0 = yield service_state.add_unit_state()
+        unit_state1 = yield service_state.add_unit_state()
+
+        group = GroupPrincipal(
+            self.client, "/services/%s/group" % service_state.internal_id)
+        members = yield group.get_members()
+
+        self.assertTrue(unit_state0.internal_id in members)
+        self.assertTrue(unit_state1.internal_id in members)
+
+        yield service_state.remove_unit_state(unit_state0)
+
+        members = yield group.get_members()
+        self.assertFalse(unit_state0.internal_id in members)
+        self.assertTrue(unit_state1.internal_id in members)
+
+    @inlineCallbacks
     def test_remove_service_unit_nonexistant(self):
         """Removing a non existant service unit, is fine."""
 
@@ -653,7 +709,9 @@
                 "wordpress", self.charm_state, dummy_constraints)
         unit_state = yield service_state.add_unit_state()
         yield service_state.remove_unit_state(unit_state)
-        yield service_state.remove_unit_state(unit_state)
+        yield self.assertFailure(
+            service_state.remove_unit_state(unit_state),
+            StateChanged)
 
     @inlineCallbacks
     def test_get_all_service_states(self):