Merge lp:~natefinch/juju-core/026-EnsureMongoServer into lp:~go-bot/juju-core/trunk

[Nate Finch]

EnsureMongoServer function for MachineAgent

Implement a method to allow the machine agent to create & run the upstart script for mongo.

https://codereview.appspot.com/53220043/

[Roger Peppe (rogpeppe)]

Looks good so far, with a couple of suggestions/queries below.

https://codereview.appspot.com/53220043/diff/20001/agent/agent.go
File agent/agent.go (right):

https://codereview.appspot.com/53220043/diff/20001/agent/agent.go#newcode232
agent/agent.go:232: func MongoDbDir(dir string) string {
or this could be a method on Config.

at the least, dir should be named "dataDir"

https://codereview.appspot.com/53220043/diff/20001/cmd/jujud/machine.go
File cmd/jujud/machine.go (right):

https://codereview.appspot.com/53220043/diff/20001/cmd/jujud/machine.go#newcode386
cmd/jujud/machine.go:386: if err := conf.Install(); err != nil {
Do we really want to do this every time the machine agent bounces?
If possible I'd prefer to avoid bouncing mongod in that case.

https://codereview.appspot.com/53220043/diff/20001/replicaset/replicaset.go
File replicaset/replicaset.go (right):

https://codereview.appspot.com/53220043/diff/20001/replicaset/replicaset.go#newcode257
replicaset/replicaset.go:257: State MemberState `bson:"State"`
"state", surely?

https://codereview.appspot.com/53220043/

[Nate Finch (natefinch)]

Reviewers: mp+201961_code.launchpad.net, rog,

Message:
Some more clean up, and now add the "ensure" part of ensure mongo
server. Also consolidated the code so we only do this in machine agent.

Also, just ignore the stuff in replicaset.go, it's not really part of
this change and I'll remove it.

Description:
EnxureMongoServer function for MachineAgent

IMplement a method to allow the machine agent to create & run the
upstart script for mongo.

https://code.launchpad.net/~natefinch/juju-core/026-EnsureMongoServer/+merge/201961

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/53220043/

Affected files (+130, -58 lines):
   A [revision details]
   M agent/agent.go
   cmd/jujud/machine.go
   M environs/cloudinit/cloudinit.go
   M provider/local/config.go
   M provider/local/environ.go
   M replicaset/replicaset.go
   M replicaset/replicaset_test.go
   M upstart/service.go

[Nate Finch (natefinch)]

Please take a look.

https://codereview.appspot.com/53220043/

[William Reade (fwereade)]

couple of quick comments, I think I may have misunderstood the StatePort
bits, can we chat when you're free?

https://codereview.appspot.com/53220043/diff/60001/agent/agent.go
File agent/agent.go (right):

https://codereview.appspot.com/53220043/diff/60001/agent/agent.go#newcode99
agent/agent.go:99: StatePort() int
I'm a bit bothered about storing this in agent config. And actually
about storing *any* of this stuff in agent config -- we have manual
bootstrap now, and I don't see why this is needed any more.

https://codereview.appspot.com/53220043/diff/60001/provider/local/environ.go
File provider/local/environ.go (right):

https://codereview.appspot.com/53220043/diff/60001/provider/local/environ.go#newcode427
provider/local/environ.go:427: journalDir :=
filepath.Join(env.config.rootDir(), "db/journal")
"db", "journal"?

https://codereview.appspot.com/53220043/diff/60001/upstart/service.go
File upstart/service.go (right):

https://codereview.appspot.com/53220043/diff/60001/upstart/service.go#newcode26
upstart/service.go:26: func MongoUpstartService(name, dataDir string,
port int) *Conf {
Hmm, I'm really not sure this is the right package for this file. Isn't
this stuff all specific to instance config?

https://codereview.appspot.com/53220043/

[lp:~natefinch/juju-core/026-EnsureMongoServer]

2185. By Nate Finch on 2014-01-23

Add code to mock out upstart so we can successfully run the new machine agent code during tests. Also added code to only run EnsureMongoServer when the machine agent has the ManageEnviron job

2186. By Nate Finch on 2014-01-28

fix a couple build errors

2187. By Nate Finch on 2014-01-28

make sure the bootstrap node doesn't try to connect to the API to get its state

2188. By Nate Finch on 2014-01-28

I should learn to hit save before commiting

Unmerged revisions

2188. By Nate Finch on 2014-01-28

I should learn to hit save before commiting

2187. By Nate Finch on 2014-01-28

make sure the bootstrap node doesn't try to connect to the API to get its state

2186. By Nate Finch on 2014-01-28

fix a couple build errors

2185. By Nate Finch on 2014-01-23

Add code to mock out upstart so we can successfully run the new machine agent code during tests. Also added code to only run EnsureMongoServer when the machine agent has the ManageEnviron job

2184. By Nate Finch on 2014-01-16

add the ensure part of ensuremongoserver, also remove the old upstart stuff so we always run it from machine agent

2183. By Nate Finch on 2014-01-16

twiddle the comment to help kick rietveld

2182. By Nate Finch on 2014-01-16

line got deleted somehow

2181. By Nate Finch on 2014-01-16

create ensuremongoserver function to create/rewrite the mongo upstart script from the machine agent

2180. By Nate Finch on 2014-01-15

Let's spell replSet correctly, eh?

2179. By Nate Finch on 2014-01-07

uncomment some tests, change mongo upstart to always set mongo in replicaset mode

=== modified file 'agent/agent.go'
--- agent/agent.go	2014-01-23 05:21:21 +0000
+++ agent/agent.go	2014-01-28 15:13:44 +0000
@@ -95,6 +95,10 @@
 	// SetValue updates the value for the specified key.
 	SetValue(key, value string)
 
+	// StatePort specifies the TCP port that will be used
+	// by the MongoDB server if one is running on this machine.
+	StatePort() int
+
 	StateInitializer
 }
 
@@ -133,6 +137,7 @@
 	stateServerKey  []byte
 	apiPort         int
 	values          map[string]string
+	statePort       int
 }
 
 type AgentConfigParams struct {
@@ -215,6 +220,7 @@
 	config.stateServerCert = params.StateServerCert
 	config.stateServerKey = params.StateServerKey
 	config.apiPort = params.APIPort
+	config.statePort = params.StatePort
 	return config, nil
 }
 
@@ -321,6 +327,10 @@
 	return Dir(c.dataDir, c.tag)
 }
 
+func (c *configInternal) StatePort() int {
+	return c.statePort
+}
+
 func (c *configInternal) check() error {
 	if c.stateDetails == nil && c.apiDetails == nil {
 		return requiredError("state or API addresses")
 
=== modified file 'cmd/jujud/machine.go'
--- cmd/jujud/machine.go	2014-01-24 14:52:58 +0000
+++ cmd/jujud/machine.go	2014-01-28 15:13:44 +0000
@@ -5,6 +5,7 @@
 
 import (
 	"fmt"
+	"io/ioutil"
 	"os"
 	"path/filepath"
 	"time"
@@ -45,15 +46,23 @@
 	"launchpad.net/juju-core/worker/upgrader"
 )
 
-var logger = loggo.GetLogger("juju.cmd.jujud")
-
-var newRunner = func(isFatal func(error) bool, moreImportant func(e0, e1 error) bool) worker.Runner {
-	return worker.NewRunner(isFatal, moreImportant)
-}
-
-const bootstrapMachineId = "0"
-
-var retryDelay = 3 * time.Second
+const (
+	mongoSvcFmt         = "juju-db-v%d"
+	bootstrapMachineId  = "0"
+	oldMongoServiceName = "juju-db"
+)
+
+var (
+	logger = loggo.GetLogger("juju.cmd.jujud")
+
+	newRunner = func(isFatal func(error) bool, moreImportant func(e0, e1 error) bool) worker.Runner {
+		return worker.NewRunner(isFatal, moreImportant)
+	}
+
+	mongoServiceName = fmt.Sprintf(mongoSvcFmt, upstart.MongoScriptVersion)
+
+	retryDelay = 3 * time.Second
+)
 
 var jujuRun = "/usr/local/bin/juju-run"
 
@@ -119,6 +128,16 @@
 		return err
 	}
 
+	mgr, err := a.isEnvironManager()
+	if err != nil {
+		return err
+	}
+	if mgr {
+		if err := a.ensureMongoServer(); err != nil {
+			return err
+		}
+	}
+
 	// ensureStateWorker ensures that there is a worker that
 	// connects to the state that runs within itself all the workers
 	// that need a state connection. Unless we're bootstrapping, we
@@ -144,10 +163,14 @@
 	a.runner.StartWorker("api", func() (worker.Worker, error) {
 		return a.APIWorker(ensureStateWorker)
 	})
+<<<<<<< TREE
 	a.runner.StartWorker("termination", func() (worker.Worker, error) {
 		return terminationworker.NewWorker(), nil
 	})
 	err := a.runner.Wait()
+=======
+	err = a.runner.Wait()
+>>>>>>> MERGE-SOURCE
 	if err == worker.ErrTerminateAgent {
 		err = a.uninstallAgent()
 	}
@@ -361,6 +384,7 @@
 	return names.MachineTag(a.MachineId)
 }
 
+<<<<<<< TREE
 func (a *MachineAgent) initAgent() error {
 	if err := os.Remove(jujuRun); err != nil && !os.IsNotExist(err) {
 		return err
@@ -369,6 +393,29 @@
 	return os.Symlink(jujud, jujuRun)
 }
 
+=======
+// isEnvironManager returns if the current agent has the ManageEnviron job.
+func (a *MachineAgent) isEnvironManager() (bool, error) {
+	if a.MachineId == bootstrapMachineId {
+		return true, nil
+	}
+
+	st, entity, err := openAPIState(a.Conf.config, a)
+	if err != nil {
+		return false, err
+	}
+	defer st.Close()
+	reportOpenedAPI(st)
+
+	for _, job := range entity.Jobs() {
+		if job == params.JobManageEnviron {
+			return true, nil
+		}
+	}
+	return false, nil
+}
+
+>>>>>>> MERGE-SOURCE
 func (a *MachineAgent) uninstallAgent() error {
 	var errors []error
 	agentServiceName := a.Conf.config.Value(agent.AgentServiceName)
@@ -381,6 +428,7 @@
 			errors = append(errors, fmt.Errorf("cannot remove service %q: %v", agentServiceName, err))
 		}
 	}
+<<<<<<< TREE
 	// Remove the rsyslog conf file and restart rsyslogd.
 	if rsyslogConfPath := a.Conf.config.Value(agent.RsyslogConfPath); rsyslogConfPath != "" {
 		if err := os.Remove(rsyslogConfPath); err != nil {
@@ -394,15 +442,16 @@
 	if err := os.Remove(jujuRun); err != nil && !os.IsNotExist(err) {
 		errors = append(errors, err)
 	}
+=======
+
+>>>>>>> MERGE-SOURCE
 	// The machine agent may terminate without knowing its jobs,
 	// for example if the machine's entry in state was removed.
 	// Thus, we do not rely on jobs here, and instead just check
 	// if the upstart config exists.
-	mongoServiceName := a.Conf.config.Value(agent.MongoServiceName)
-	if mongoServiceName != "" {
-		if err := upstart.NewService(mongoServiceName).StopAndRemove(); err != nil {
-			errors = append(errors, fmt.Errorf("cannot stop/remove service %q: %v", mongoServiceName, err))
-		}
+	mongo := a.mongoService()
+	if err := mongo.StopAndRemove(); err != nil {
+		errors = append(errors, fmt.Errorf("cannot stop/remove service %q: %v", mongo.Name, err))
 	}
 	if err := os.RemoveAll(a.Conf.dataDir); err != nil {
 		errors = append(errors, err)
@@ -413,6 +462,79 @@
 	return fmt.Errorf("uninstall failed: %v", errors)
 }
 
+// ensureMongoServer ensures that the correct mongo upstart script is installed
+// and running.
+//
+// This method will remove old versions of the mongo upstart script as necessary
+// before installing the new version.
+func (a *MachineAgent) ensureMongoServer() error {
+	service := a.mongoService()
+	if service.Installed() {
+		return nil
+	}
+
+	if err := removeOldMongoServices(); err != nil {
+		return err
+	}
+
+	journalDir := filepath.Join(a.mongoDir(), "journal")
+
+	if err := os.MkdirAll(journalDir, 0700); err != nil {
+		logger.Errorf("failed to make mongo journal dir %s: %v", journalDir, err)
+		return err
+	}
+
+	// manually create the prealloc files, since otherwise they get created as 100M files.
+	zeroes := make([]byte, 1024*1024)
+	for x := 0; x < 3; x++ {
+		name := fmt.Sprintf("prealloc.%d", x)
+		filename := filepath.Join(journalDir, name)
+		if err := ioutil.WriteFile(filename, zeroes, 700); err != nil {
+			logger.Errorf("failed to make write mongo prealloc file: %v", journalDir, err)
+			return err
+		}
+	}
+
+	if err := service.Install(); err != nil {
+		logger.Errorf("Failed to install mongo service %q: %v", service.Name, err)
+		return err
+	}
+	return service.Start()
+}
+
+// mongoDir returns the directory that mongo should use to store its data.
+func (a *MachineAgent) mongoDir() string {
+	return a.Conf.dataDir
+}
+
+// mongoSvcConfig returns the upstart configuration object for mongo.
+func (a *MachineAgent) mongoService() *upstart.Conf {
+	return upstart.MongoUpstartService(
+		mongoServiceName,
+		a.mongoDir(),
+		a.Conf.config.StatePort())
+}
+
+// removeOldMongoServices looks for any old juju mongo upstart scripts and
+// removes them.
+func removeOldMongoServices() error {
+	old := upstart.NewService(oldMongoServiceName)
+	if err := old.StopAndRemove(); err != nil {
+		logger.Errorf("Failed to remove old mongo upstart service %q: %v", old.Name, err)
+		return err
+	}
+
+	// the new formatting for the script name started at version 2
+	for x := 2; x < upstart.MongoScriptVersion; x++ {
+		old := upstart.NewService(fmt.Sprintf(mongoSvcFmt, x))
+		if err := old.StopAndRemove(); err != nil {
+			logger.Errorf("Failed to remove old mongo upstart service %q: %v", old.Name, err)
+			return err
+		}
+	}
+	return nil
+}
+
 // Below pieces are used for testing,to give us access to the *State opened
 // by the agent, and allow us to trigger syncs without waiting 5s for them
 // to happen automatically.
 
=== modified file 'cmd/jujud/machine_test.go'
--- cmd/jujud/machine_test.go	2014-01-24 14:52:58 +0000
+++ cmd/jujud/machine_test.go	2014-01-28 15:13:44 +0000
@@ -29,6 +29,7 @@
 	charmtesting "launchpad.net/juju-core/state/apiserver/charmrevisionupdater/testing"
 	statetesting "launchpad.net/juju-core/state/testing"
 	"launchpad.net/juju-core/state/watcher"
+	"launchpad.net/juju-core/testing"
 	coretesting "launchpad.net/juju-core/testing"
 	jc "launchpad.net/juju-core/testing/checkers"
 	"launchpad.net/juju-core/testing/testbase"
@@ -44,6 +45,7 @@
 type MachineSuite struct {
 	agentSuite
 	lxctesting.TestSuite
+	testing.MockUpstartSuite
 }
 
 var _ = gc.Suite(&MachineSuite{})
@@ -51,6 +53,7 @@
 func (s *MachineSuite) SetUpSuite(c *gc.C) {
 	s.agentSuite.SetUpSuite(c)
 	s.TestSuite.SetUpSuite(c)
+	s.MockUpstartSuite.SetUpSuite(c)
 	restore := testbase.PatchValue(&charm.CacheDir, c.MkDir())
 	s.AddSuiteCleanup(func(*gc.C) { restore() })
 }
@@ -58,21 +61,27 @@
 func (s *MachineSuite) TearDownSuite(c *gc.C) {
 	s.TestSuite.TearDownSuite(c)
 	s.agentSuite.TearDownSuite(c)
+	s.MockUpstartSuite.TearDownSuite(c)
 }
 
 func (s *MachineSuite) SetUpTest(c *gc.C) {
 	s.agentSuite.SetUpTest(c)
 	s.TestSuite.SetUpTest(c)
+<<<<<<< TREE
 	os.Remove(jujuRun) // ignore error; may not exist
 	// Fake $HOME, and ssh user to avoid touching ~ubuntu/.ssh/authorized_keys.
 	fakeHome := coretesting.MakeEmptyFakeHomeWithoutJuju(c)
 	s.AddCleanup(func(*gc.C) { fakeHome.Restore() })
 	s.PatchValue(&authenticationworker.SSHUser, "")
+=======
+	s.MockUpstartSuite.SetUpTest(c)
+>>>>>>> MERGE-SOURCE
 }
 
 func (s *MachineSuite) TearDownTest(c *gc.C) {
 	s.TestSuite.TearDownTest(c)
 	s.agentSuite.TearDownTest(c)
+	s.MockUpstartSuite.TearDownTest(c)
 }
 
 const initialMachinePassword = "machine-password-1234567890"
 
=== modified file 'environs/cloudinit/cloudinit.go'
--- environs/cloudinit/cloudinit.go	2014-01-26 23:28:43 +0000
+++ environs/cloudinit/cloudinit.go	2014-01-28 15:13:44 +0000
@@ -352,9 +352,7 @@
 		}
 		certKey := string(cfg.StateServerCert) + string(cfg.StateServerKey)
 		c.AddFile(cfg.dataFile("server.pem"), certKey, 0600)
-		if err := cfg.addMongoToBoot(c); err != nil {
-			return err
-		}
+
 		// We temporarily give bootstrap-state a directory
 		// of its own so that it can get the state info via the
 		// same mechanism as other jujud commands.
@@ -449,11 +447,16 @@
 	if err != nil {
 		return nil, err
 	}
+<<<<<<< TREE
 	acfg.SetValue(agent.RsyslogConfPath, cfg.RsyslogConfPath)
 	acfg.SetValue(agent.AgentServiceName, cfg.MachineAgentServiceName)
 	if cfg.StateServer {
 		acfg.SetValue(agent.MongoServiceName, cfg.MongoServiceName)
 	}
+=======
+	acfg.SetValue(agent.AgentServiceName, machineAgentServiceName(tag))
+
+>>>>>>> MERGE-SOURCE
 	cmds, err := acfg.WriteCommands()
 	if err != nil {
 		return nil, err
@@ -462,6 +465,13 @@
 	return acfg, nil
 }
 
+<<<<<<< TREE
+=======
+func machineAgentServiceName(tag string) string {
+	return "jujud-" + tag
+}
+
+>>>>>>> MERGE-SOURCE
 func (cfg *MachineConfig) addMachineAgentToBoot(c *cloudinit.Config, tag, machineId string) error {
 	// Make the agent run via a symbolic link to the actual tools
 	// directory, so it can upgrade itself without needing to change
@@ -481,6 +491,7 @@
 	return nil
 }
 
+<<<<<<< TREE
 func (cfg *MachineConfig) addMongoToBoot(c *cloudinit.Config) error {
 	dbDir := path.Join(cfg.DataDir, "db")
 	c.AddScripts(
@@ -503,6 +514,8 @@
 	return nil
 }
 
+=======
+>>>>>>> MERGE-SOURCE
 // versionDir converts a tools URL into a name
 // to use as a directory for storing the tools executables in
 // by using the last element stripped of its extension.
 
=== modified file 'provider/local/config.go'
--- provider/local/config.go	2014-01-24 01:50:07 +0000
+++ provider/local/config.go	2014-01-28 15:13:44 +0000
@@ -81,10 +81,6 @@
 	return filepath.Join(c.rootDir(), "storage")
 }
 
-func (c *environConfig) mongoDir() string {
-	return filepath.Join(c.rootDir(), "db")
-}
-
 func (c *environConfig) logDir() string {
 	return filepath.Join(c.rootDir(), "log")
 }
@@ -127,7 +123,6 @@
 	for _, dirname := range []string{
 		c.sharedStorageDir(),
 		c.storageDir(),
-		c.mongoDir(),
 		c.logDir(),
 	} {
 		logger.Tracef("creating directory %s", dirname)
 
=== modified file 'provider/local/environ.go'
--- provider/local/environ.go	2014-01-24 14:52:58 +0000
+++ provider/local/environ.go	2014-01-28 15:13:44 +0000
@@ -449,3 +449,213 @@
 func (env *localEnviron) Provider() environs.EnvironProvider {
 	return providerInstance
 }
+<<<<<<< TREE
+=======
+
+// setupLocalMongoService returns the cert and key if there was no error.
+func (env *localEnviron) setupLocalMongoService() ([]byte, []byte, error) {
+	journalDir := filepath.Join(env.config.rootDir(), "db/journal")
+	logger.Debugf("create mongo journal dir: %v", journalDir)
+	if err := os.MkdirAll(journalDir, 0755); err != nil {
+		logger.Errorf("failed to make mongo journal dir %s: %v", journalDir, err)
+		return nil, nil, err
+	}
+
+	logger.Debugf("generate server cert")
+	cert, key, err := env.config.GenerateStateServerCertAndKey()
+	if err != nil {
+		logger.Errorf("failed to generate server cert: %v", err)
+		return nil, nil, err
+	}
+	if err := ioutil.WriteFile(
+		env.config.configFile("server.pem"),
+		append(cert, key...),
+		0600); err != nil {
+		logger.Errorf("failed to write server.pem: %v", err)
+		return nil, nil, err
+	}
+
+	mongo := upstart.MongoUpstartService(
+		env.mongoServiceName(),
+		env.config.rootDir(),
+		env.config.StatePort())
+	mongo.InitDir = upstartScriptLocation
+	logger.Infof("installing service %s to %s", env.mongoServiceName(), mongo.InitDir)
+	if err := mongo.Install(); err != nil {
+		logger.Errorf("could not install mongo service: %v", err)
+		return nil, nil, err
+	}
+	return cert, key, nil
+}
+
+func (env *localEnviron) setupLocalMachineAgent(cons constraints.Value, possibleTools tools.List) error {
+	dataDir := env.config.rootDir()
+	// unpack the first tools into the agent dir.
+	agentTools := possibleTools[0]
+	logger.Debugf("tools: %#v", agentTools)
+	// brutally abuse our knowledge of storage to directly open the file
+	toolsUrl, err := url.Parse(agentTools.URL)
+	if err != nil {
+		return err
+	}
+	toolsLocation := filepath.Join(env.config.storageDir(), toolsUrl.Path)
+	logger.Infof("tools location: %v", toolsLocation)
+	toolsFile, err := os.Open(toolsLocation)
+	defer toolsFile.Close()
+	// Again, brutally abuse our knowledge here.
+
+	// The tools that possible bootstrap tools are based on the
+	// default series in the config.  However we are running potentially on a
+	// different series.  When the machine agent is started, it will be
+	// looking based on the current series, so we need to override the series
+	// returned in the tools to be the current series.
+	agentTools.Version.Series = version.Current.Series
+	err = agenttools.UnpackTools(dataDir, agentTools, toolsFile)
+
+	machineId := "0" // Always machine 0
+	tag := names.MachineTag(machineId)
+
+	// make sure we create the symlink so we have it for the upstart config to use
+	if _, err := agenttools.ChangeAgentTools(dataDir, tag, agentTools.Version); err != nil {
+		logger.Errorf("could not create tools directory symlink: %v", err)
+		return err
+	}
+
+	toolsDir := agenttools.ToolsDir(dataDir, tag)
+
+	logDir := env.config.logDir()
+	machineEnvironment := map[string]string{
+		"USER": env.config.user,
+		"HOME": osenv.Home(),
+	}
+	agentService := upstart.MachineAgentUpstartService(
+		env.machineAgentServiceName(),
+		toolsDir, dataDir, logDir, tag, machineId, machineEnvironment)
+
+	agentService.InitDir = upstartScriptLocation
+	logger.Infof("installing service %s to %s", env.machineAgentServiceName(), agentService.InitDir)
+	if err := agentService.Install(); err != nil {
+		logger.Errorf("could not install machine agent service: %v", err)
+		return err
+	}
+	return nil
+}
+
+func (env *localEnviron) findBridgeAddress(networkBridge string) (string, error) {
+	return getAddressForInterface(networkBridge)
+}
+
+func (env *localEnviron) writeBootstrapAgentConfFile(secret string, cert, key []byte) (agent.Config, error) {
+	tag := names.MachineTag("0")
+	passwordHash := utils.UserPasswordHash(secret, utils.CompatSalt)
+	// We don't check the existance of the CACert here as if it wasn't set, we
+	// wouldn't get this far.
+	cfg := env.config.Config
+	caCert, _ := cfg.CACert()
+	agentValues := map[string]string{
+		agent.ProviderType:      env.config.Type(),
+		agent.Namespace:         env.config.namespace(),
+		agent.StorageDir:        env.config.storageDir(),
+		agent.StorageAddr:       env.config.storageAddr(),
+		agent.SharedStorageDir:  env.config.sharedStorageDir(),
+		agent.SharedStorageAddr: env.config.sharedStorageAddr(),
+		agent.AgentServiceName:  env.machineAgentServiceName(),
+		agent.MongoServiceName:  env.mongoServiceName(),
+	}
+	// NOTE: the state address HAS to be localhost, otherwise the mongo
+	// initialization fails.  There is some magic code somewhere in the mongo
+	// connection code that treats connections from localhost as special, and
+	// will raise unauthorized errors during the initialization if the caller
+	// is not connected from localhost.
+	stateAddress := fmt.Sprintf("localhost:%d", cfg.StatePort())
+	apiAddress := fmt.Sprintf("localhost:%d", cfg.APIPort())
+	config, err := agent.NewStateMachineConfig(
+		agent.StateMachineConfigParams{
+			AgentConfigParams: agent.AgentConfigParams{
+				DataDir:        env.config.rootDir(),
+				Tag:            tag,
+				Password:       passwordHash,
+				Nonce:          state.BootstrapNonce,
+				StateAddresses: []string{stateAddress},
+				APIAddresses:   []string{apiAddress},
+				CACert:         caCert,
+				Values:         agentValues,
+			},
+			StateServerCert: cert,
+			StateServerKey:  key,
+			StatePort:       cfg.StatePort(),
+			APIPort:         cfg.APIPort(),
+		})
+	if err != nil {
+		return nil, err
+	}
+	if err := config.Write(); err != nil {
+		logger.Errorf("failed to write bootstrap agent file: %v", err)
+		return nil, err
+	}
+	return config, nil
+}
+
+func (env *localEnviron) initializeState(agentConfig agent.Config, cons constraints.Value) error {
+	bootstrapCfg, err := environs.BootstrapConfig(env.config.Config)
+	if err != nil {
+		return err
+	}
+	st, m, err := agentConfig.InitializeState(bootstrapCfg, agent.BootstrapMachineConfig{
+		Constraints: cons,
+		Jobs: []state.MachineJob{
+			state.JobManageEnviron,
+			state.JobManageState,
+		},
+		InstanceId: bootstrapInstanceId,
+	}, state.DialOpts{
+		Timeout: 60 * time.Second,
+	})
+	if err != nil {
+		return err
+	}
+	defer st.Close()
+	addr, err := env.findBridgeAddress(env.config.networkBridge())
+	if err != nil {
+		return fmt.Errorf("failed to get bridge address: %v", err)
+	}
+	err = m.SetAddresses([]instance.Address{{
+		NetworkScope: instance.NetworkPublic,
+		Type:         instance.HostName,
+		Value:        "localhost",
+	}, {
+		NetworkScope: instance.NetworkCloudLocal,
+		Type:         instance.Ipv4Address,
+		Value:        addr,
+	}})
+	if err != nil {
+		return fmt.Errorf("cannot set addresses on bootstrap instance: %v", err)
+	}
+	return nil
+}
+
+func (env *localEnviron) configureLocalSyslog() error {
+	tag := names.MachineTag("0")
+	syslogConfigRenderer := syslog.NewAccumulateConfig(tag, env.config.SyslogPort(), env.config.namespace())
+	syslogConfigRenderer.ConfigDir = syslogConfigDir
+	syslogConfigRenderer.ConfigFileName = env.syslogFilename()
+	syslogConfigRenderer.LogDir = env.config.logDir()
+	if err := syslogConfigRenderer.Write(); err != nil {
+		return err
+	}
+	if err := syslog.Restart(); err != nil {
+		logger.Warningf("cannot restart syslog daemon: %v", err)
+	}
+	return nil
+}
+
+func (env *localEnviron) removeLocalSyslog() {
+	// Don't fail if we have issues, but warn the user.
+	if err := os.Remove(filepath.Join(syslogConfigDir, env.syslogFilename())); err != nil {
+		logger.Warningf("could not remove local syslog config: %v", err)
+	}
+	if err := syslog.Restart(); err != nil {
+		logger.Warningf("cannot restart syslog daemon: %v", err)
+	}
+}
+>>>>>>> MERGE-SOURCE
 
=== modified file 'replicaset/replicaset.go'
--- replicaset/replicaset.go	2014-01-21 19:26:14 +0000
+++ replicaset/replicaset.go	2014-01-28 15:13:44 +0000
@@ -275,7 +275,11 @@
 	Healthy bool `bson:"health"`
 
 	// State describes the current state of the member.
+<<<<<<< TREE
 	State MemberState `bson:"state"`
+=======
+	State MemberState `bson:"State"`
+>>>>>>> MERGE-SOURCE
 
 	// Uptime describes how long the member has been online.
 	Uptime time.Duration `bson:"uptime"`
 
=== modified file 'replicaset/replicaset_test.go'
=== added file 'testing/mockupstart.go'
--- testing/mockupstart.go	1970-01-01 00:00:00 +0000
+++ testing/mockupstart.go	2014-01-28 15:13:44 +0000
@@ -0,0 +1,42 @@
+package testing
+
+import (
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+
+	gc "launchpad.net/gocheck"
+
+	"launchpad.net/juju-core/testing/testbase"
+	"launchpad.net/juju-core/upstart"
+)
+
+// MockUpstartSuite is a testing suite that you can embed to easily mock out
+// upstart.
+type MockUpstartSuite struct {
+	testbase.CleanupSuite
+	binDir string
+}
+
+var _ = gc.Suite(&MockUpstartSuite{})
+
+func (s *MockUpstartSuite) SetUpSuite(c *gc.C) {
+	s.CleanupSuite.SetUpSuite(c)
+	s.binDir = c.MkDir()
+	s.PatchEnvironment("PATH", fmt.Sprintf("%v%v%v", s.binDir, filepath.ListSeparator, os.Getenv("PATH")))
+	s.PatchValue(&upstart.InitDir, c.MkDir())
+	s.makeBin(c, "status", `echo "blah stop/waiting"`)
+	s.makeBin(c, "stopped-status", `echo "blah stop/waiting"`)
+	s.makeBin(c, "started-status", `echo "blah start/running, process 666"`)
+	s.makeBin(c, "start", "cp $(which started-status) $(which status)")
+	s.makeBin(c, "stop", "cp $(which stopped-status) $(which status)")
+}
+
+// makeBin makes a binary with the given name and contents in the binDir.  This
+// is an easy way to mock the behavior of built-in applications.
+func (s *MockUpstartSuite) makeBin(c *gc.C, name, script string) {
+	path := filepath.Join(s.binDir, name)
+	err := ioutil.WriteFile(path, []byte("#!/bin/bash --norc\n"+script), 0755)
+	c.Assert(err, gc.IsNil)
+}
 
=== modified file 'upstart/service.go'
--- upstart/service.go	2013-11-19 08:53:40 +0000
+++ upstart/service.go	2014-01-28 15:13:44 +0000
@@ -13,12 +13,22 @@
 const (
 	maxMongoFiles = 65000
 	maxAgentFiles = 20000
+
+	// MongoScriptVersion keeps track of changes to the mongo upstart script.
+	// Update this version when you update the script that gets installed from
+	// MongoUpstartService.
+	MongoScriptVersion = 2
 )
 
 // MongoUpstartService returns the upstart config for the mongo state service.
-func MongoUpstartService(name, dataDir, dbDir string, port int) *Conf {
+//
+// This method assumes there is a server.pem keyfile in dataDir.
+func MongoUpstartService(name, dataDir string, port int) *Conf {
 	keyFile := path.Join(dataDir, "server.pem")
 	svc := NewService(name)
+
+	dbDir := path.Join(dataDir, "db")
+
 	return &Conf{
 		Service: *svc,
 		Desc:    "juju state database",
@@ -36,7 +46,8 @@
 			" --port " + fmt.Sprint(port) +
 			" --noprealloc" +
 			" --syslog" +
-			" --smallfiles",
+			" --smallfiles" +
+			" --replSet juju",
 	}
 }
 
 
=== modified file 'upstart/upstart.go'
--- upstart/upstart.go	2013-10-02 03:44:40 +0000
+++ upstart/upstart.go	2014-01-28 15:13:44 +0000
@@ -18,7 +18,13 @@
 	"launchpad.net/juju-core/utils"
 )
 
-var startedRE = regexp.MustCompile(`^.* start/running, process (\d+)\n$`)
+var (
+	startedRE = regexp.MustCompile(`^.* start/running, process (\d+)\n$`)
+
+	// InitDir is the directory where upstart scripts will be created by
+	// default.
+	InitDir = "/etc/init"
+)
 
 var InstallStartRetryAttempts = utils.AttemptStrategy{
 	Total: 1 * time.Second,
@@ -32,7 +38,7 @@
 }
 
 func NewService(name string) *Service {
-	return &Service{Name: name, InitDir: "/etc/init"}
+	return &Service{Name: name, InitDir: InitDir}
 }
 
 // confPath returns the path to the service's configuration file.