Merge lp:~smoser/pyjuju/cloud-init-output-log into lp:pyjuju

Proposed by Scott Moser on 2011-08-31
Status: Merged
Approved by: Gustavo Niemeyer on 2011-09-12
Approved revision: 338
Merged at revision: 341
Proposed branch: lp:~smoser/pyjuju/cloud-init-output-log
Merge into: lp:pyjuju
Diff against target: 37 lines (+4/-1)
3 files modified
ensemble/providers/common/utils.py (+2/-1)
ensemble/providers/orchestra/tests/data/bootstrap_late_command (+1/-0)
ensemble/providers/orchestra/tests/data/launch_late_command (+1/-0)
To merge this branch: bzr merge lp:~smoser/pyjuju/cloud-init-output-log
Reviewer Review Type Date Requested Status
Gustavo Niemeyer Approve on 2011-09-12
Clint Byrum (community) 2011-08-31 Needs Fixing on 2011-09-06
Review via email: mp+73596@code.launchpad.net

Description of the change

log output from cloud-init and its subprocess to a file

By default cloud-init only outputs to its stdout and stderr.
In the case of EC2, that is to a console that you can get at
with get-console-output.

However, that is less than convienent as it is delayed by
4 minutes, and on other platforms, the console output isn't so easily
available.

This will just log stdout and stderr to /var/log/cloud-init-output.log

To post a comment you must log in.
Kapil Thangavelu (hazmat) wrote :

Excerpts from Scott Moser's message of Wed Aug 31 21:18:33 UTC 2011:
> Scott Moser has proposed merging lp:~smoser/ensemble/cloud-init-output-log into lp:ensemble.
>
> Requested reviews:
> Ensemble Team (ensemble)
>
> For more details, see:
> https://code.launchpad.net/~smoser/ensemble/cloud-init-output-log/+merge/73596
>
> log output from cloud-init and its subprocess to a file
>
> By default cloud-init only outputs to its stdout and stderr.
> In the case of EC2, that is to a console that you can get at
> with get-console-output.
>
> However, that is less than convienent as it is delayed by
> 4 minutes, and on other platforms, the console output isn't so easily
> available.
>
> This will just log stdout and stderr to /var/log/cloud-init-output.log
>

fwiw, this will probably cause some minor breakage of the tests in the tree as
they check cloud init formatting exactly.

Clint Byrum (clint-fewbar) wrote :
Download full text (12.4 KiB)

This breaks the test suite, possibly because it is quite rigid about what it expects to be fed to cloud-init.

[ERROR]
Traceback (most recent call last):
  File "/«BUILDDIR»/ensemble-0.5+bzr336/ensemble/lib/mocker.py", line 161, in test_method_wrapper
    self.mocker.verify()
  File "/«BUILDDIR»/ensemble-0.5+bzr336/ensemble/lib/mocker.py", line 579, in verify
    raise AssertionError(os.linesep.join(message))
exceptions.AssertionError: [Mocker] Unmet expectations:

=> proxy_m.callRemote('modify_system', 'smith', 'ks_meta', MATCH(<function verify at 0x32cb5f0>), 'TOKEN')
 - Performed fewer times than expected.

ensemble.providers.orchestra.tests.test_bootstrap.OrchestraBootstrapTest.test_cannot_modify_machine
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/trial/unittest.py", line 402, in _eb
    raise self.failureException(output)
twisted.trial.unittest.FailTest:
Expected: (<class 'ensemble.errors.ProviderError'>,)
Got:
[Failure instance: Traceback: <type 'exceptions.AssertionError'>: [Mocker] Unmet expectation:

=> not equal:
 a = '\nseed_d=/var/lib/cloud/seed/nocloud-net\nmkdir -p "$seed_d"\ncat > "$seed_d"/meta-data <<"EOF"\ninstance-id: winston-uid\nEOF\n\ncat > "$seed_d"/user-data <<"EOF"\n#cloud-config\napt-update: true\napt-upgrade: true\napt_sources:\n- {source: \'ppa:ensemble/ppa\'}\nmachine-data: {ensemble-zookeeper-hosts: \'localhost:2181\', machine-id: \'0\'}\noutput: {all: \'| tee -a /var/log/cloud-init-output.log\'}\npackages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-argparse, python-txaws,\n python-zookeeper, bzr, default-jre-headless, zookeeper, zookeeperd]\nruncmd: [export ENSEMBLE_INSTANCE_ID=winston-uid, sudo apt-get -y install ensemble,\n sudo mkdir -p /var/lib/ensemble, sudo mkdir -p /var/log/ensemble, \'ensemble-admin\n initialize --instance-id=$ENSEMBLE_INSTANCE_ID --admin-identity="admin:qRBXC1ubEEUqRL6wcBhgmc9xkaY="\',\n \'ENSEMBLE_MACHINE_ID=0 ENSEMBLE_ZOOKEEPER=localhost:2181 python -m ensemble.agents.machine\n -n --logfile=/var/log/ensemble/machine-agent.log --pidfile=/var/run/ensemble/machine-agent.pid\',\n \'ENSEMBLE_ZOOKEEPER=localhost:2181 python -m ensemble.agents.provision -n --logfile=/var/log/ensemble/provision-agent.log\n --pidfile=/var/run/ensemble/provision-agent.pid\']\nssh_authorized_keys: [this-is-a-public-key]\n\nEOF\n'
 b = '\nseed_d=/var/lib/cloud/seed/nocloud-net\nmkdir -p "$seed_d"\ncat > "$seed_d"/meta-data <<"EOF"\ninstance-id: winston-uid\nEOF\n\ncat > "$seed_d"/user-data <<"EOF"\n#cloud-config\napt-update: true\napt-upgrade: true\napt_sources:\n- {source: \'ppa:ensemble/ppa\'}\nmachine-data: {ensemble-zookeeper-hosts: \'localhost:2181\', machine-id: \'0\'}\npackages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-argparse, python-txaws,\n python-zookeeper, bzr, default-jre-headless, zookeeper, zookeeperd]\nruncmd: [export ENSEMBLE_INSTANCE_ID=winston-uid, sudo apt-get -y install ensemble,\n sudo mkdir -p /var/lib/ensemble, sudo mkdir -p /var/log/ensemble, \'ensemble-admin\n initialize --instance-id=$ENSEMBLE_INSTANCE...

review: Needs Fixing
338. By Scott Moser on 2011-09-07

make tests pass

Gustavo Niemeyer (niemeyer) wrote :

The magic in there goes definitely beyond my knowledge of cloud-init,
but I trust your take on it, thanks Scott.

review: Approve
Gustavo Niemeyer (niemeyer) wrote :

Kapil has tested and it seems to work for real as well.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'ensemble/providers/common/utils.py'
2--- ensemble/providers/common/utils.py 2011-08-23 09:59:59 +0000
3+++ ensemble/providers/common/utils.py 2011-09-07 22:09:52 +0000
4@@ -105,7 +105,8 @@
5 "apt-update": True,
6 "apt-upgrade": True,
7 "ssh_authorized_keys": [authorized_keys],
8- "packages": []}
9+ "packages": [],
10+ "output": { "all": "| tee -a /var/log/cloud-init-output.log" }}
11
12 if data:
13 cloud_config["machine-data"] = data
14
15=== modified file 'ensemble/providers/orchestra/tests/data/bootstrap_late_command'
16--- ensemble/providers/orchestra/tests/data/bootstrap_late_command 2011-08-11 18:43:06 +0000
17+++ ensemble/providers/orchestra/tests/data/bootstrap_late_command 2011-09-07 22:09:52 +0000
18@@ -12,6 +12,7 @@
19 apt_sources:
20 - {source: 'ppa:ensemble/ppa'}
21 machine-data: {ensemble-zookeeper-hosts: 'localhost:2181', machine-id: '0'}
22+output: {all: '| tee -a /var/log/cloud-init-output.log'}
23 packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-argparse, python-txaws,
24 python-zookeeper, bzr, default-jre-headless, zookeeper, zookeeperd]
25 runcmd: [export ENSEMBLE_INSTANCE_ID=winston-uid, sudo apt-get -y install ensemble,
26
27=== modified file 'ensemble/providers/orchestra/tests/data/launch_late_command'
28--- ensemble/providers/orchestra/tests/data/launch_late_command 2011-08-17 11:12:29 +0000
29+++ ensemble/providers/orchestra/tests/data/launch_late_command 2011-09-07 22:09:52 +0000
30@@ -12,6 +12,7 @@
31 apt_sources:
32 - {source: 'ppa:ensemble/ppa'}
33 machine-data: {ensemble-zookeeper-hosts: 'jennifer:2181', machine-id: '42'}
34+output: {all: '| tee -a /var/log/cloud-init-output.log'}
35 packages: [bzr, byobu, tmux, python-setuptools, python-twisted, python-argparse, python-txaws,
36 python-zookeeper]
37 runcmd: [export ENSEMBLE_INSTANCE_ID=winston-uid, sudo apt-get -y install ensemble,

Subscribers

People subscribed via source and target branches

to status/vote changes: