Merge ~ajorgens/cloud-init:octal-modes into cloud-init:master

Proposed by Andrew Jorgensen
Status: Merged
Approved by: Scott Moser
Approved revision: 2f7132a0fa8ae12f9771b5e3c44f93ef1685ea75
Merged at revision: 0fe6a0607408d387f4b0d4482b95afbc5d3f3909
Proposed branch: ~ajorgens/cloud-init:octal-modes
Merge into: cloud-init:master
Diff against target: 146 lines (+52/-9)
6 files modified
cloudinit/config/cc_write_files.py (+9/-1)
cloudinit/util.py (+5/-1)
tests/unittests/helpers.py (+5/-3)
tests/unittests/test_datasource/test_azure.py (+2/-1)
tests/unittests/test_handler/test_handler_ntp.py (+2/-1)
tests/unittests/test_handler/test_handler_write_files.py (+29/-2)
Reviewer Review Type Date Requested Status
Scott Moser Approve
Server Team CI bot continuous-integration Approve
Chad Smith Approve
Andrew Jorgensen (community) Needs Resubmitting
Review via email: mp+325424@code.launchpad.net

Commit message

write_file(s): Print file modes as octal, not decimal

Unix file modes are usually represented as octal, but they were being
interpreted as decimal, for example 0o644 would be printed as '420'.

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Andrew Jorgensen (ajorgens) wrote :

Well, dang, I thought this was going to be the easy one to get merged. Why is the mode None? *sigh*

Revision history for this message
Scott Moser (smoser) wrote :

Andrew,

Thanks for re-submitting this MP.

I've reviewed a little bit and found a few things
a.) as seen in the c-i logs, the change to 'decode_perms' can fail if the input was a string.
  >>> "%o" % "0755"
  TypeError: %o format: an integer is required, not str

b.) it seems to have exposed an error in append_file() that i'm not sure how it ever worked.

For your MP to be accepted, we need to fix 'a'

I've put a suggested change like:
 http://paste.ubuntu.com/24841624/

Basically that just tries to do '%o' but uses '%r' if that fails.

Thoughts?

review: Needs Fixing
~ajorgens/cloud-init:octal-modes updated
5a9acd1... by Andrew Jorgensen

Additional fixes and test case from smoser@

Revision history for this message
Andrew Jorgensen (ajorgens) wrote :

Both a and b are fixed in 5a9acd1

review: Needs Resubmitting
Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

Thanks, this looks good to me.
I'm asking Chad for a quick review also.

review: Approve
Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Chad Smith (chad.smith) wrote :

Looks good and thanks for the unit tests. Since we now have a with_logs option in CiTest class, let's use that instead of mocks.

Here's a slight extension to our unit test logging setup to also make sure we check log level on some of these logs.
http://paste.ubuntu.com/24843185/

Approved if folks think this is a good idea with the following patch.

review: Approve
~ajorgens/cloud-init:octal-modes updated
2f7132a... by Andrew Jorgensen

Unit test fixes from csmith and smoser

Revision history for this message
Andrew Jorgensen (ajorg) wrote :

Ready to merge AFAICT.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

I approve this.
We really ought to change the signature of decode_perms to not take a 'log' input but rather just to use logging.
Andrew, interested in doing a follow up for that?

review: Approve
Revision history for this message
Andrew Jorgensen (ajorgens) wrote :

> Andrew, interested in doing a follow up for that?

Probably. I've got a rather long queue of patches to post, but I've added this to the TODO list.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/cloudinit/config/cc_write_files.py b/cloudinit/config/cc_write_files.py
index 72e1cdd..1835a31 100644
--- a/cloudinit/config/cc_write_files.py
+++ b/cloudinit/config/cc_write_files.py
@@ -53,6 +53,7 @@ import six
53from cloudinit.settings import PER_INSTANCE53from cloudinit.settings import PER_INSTANCE
54from cloudinit import util54from cloudinit import util
5555
56
56frequency = PER_INSTANCE57frequency = PER_INSTANCE
5758
58DEFAULT_OWNER = "root:root"59DEFAULT_OWNER = "root:root"
@@ -119,7 +120,14 @@ def decode_perms(perm, default, log):
119 # Force to string and try octal conversion120 # Force to string and try octal conversion
120 return int(str(perm), 8)121 return int(str(perm), 8)
121 except (TypeError, ValueError):122 except (TypeError, ValueError):
122 log.warn("Undecodable permissions %s, assuming %s", perm, default)123 reps = []
124 for r in (perm, default):
125 try:
126 reps.append("%o" % r)
127 except TypeError:
128 reps.append("%r" % r)
129 log.warning(
130 "Undecodable permissions {0}, returning default {1}".format(*reps))
123 return default131 return default
124132
125133
diff --git a/cloudinit/util.py b/cloudinit/util.py
index 415ca37..ec68925 100644
--- a/cloudinit/util.py
+++ b/cloudinit/util.py
@@ -1751,8 +1751,12 @@ def write_file(filename, content, mode=0o644, omode="wb", copy_mode=False):
1751 else:1751 else:
1752 content = decode_binary(content)1752 content = decode_binary(content)
1753 write_type = 'characters'1753 write_type = 'characters'
1754 try:
1755 mode_r = "%o" % mode
1756 except TypeError:
1757 mode_r = "%r" % mode
1754 LOG.debug("Writing to %s - %s: [%s] %s %s",1758 LOG.debug("Writing to %s - %s: [%s] %s %s",
1755 filename, omode, mode, len(content), write_type)1759 filename, omode, mode_r, len(content), write_type)
1756 with SeLinuxGuard(path=filename):1760 with SeLinuxGuard(path=filename):
1757 with open(filename, omode) as fh:1761 with open(filename, omode) as fh:
1758 fh.write(content)1762 fh.write(content)
diff --git a/tests/unittests/helpers.py b/tests/unittests/helpers.py
index e78abce..569f1ae 100644
--- a/tests/unittests/helpers.py
+++ b/tests/unittests/helpers.py
@@ -97,11 +97,13 @@ class CiTestCase(TestCase):
97 super(CiTestCase, self).setUp()97 super(CiTestCase, self).setUp()
98 if self.with_logs:98 if self.with_logs:
99 # Create a log handler so unit tests can search expected logs.99 # Create a log handler so unit tests can search expected logs.
100 logger = logging.getLogger()100 self.logger = logging.getLogger()
101 self.logs = six.StringIO()101 self.logs = six.StringIO()
102 formatter = logging.Formatter('%(levelname)s: %(message)s')
102 handler = logging.StreamHandler(self.logs)103 handler = logging.StreamHandler(self.logs)
103 self.old_handlers = logger.handlers104 handler.setFormatter(formatter)
104 logger.handlers = [handler]105 self.old_handlers = self.logger.handlers
106 self.logger.handlers = [handler]
105107
106 def tearDown(self):108 def tearDown(self):
107 if self.with_logs:109 if self.with_logs:
diff --git a/tests/unittests/test_datasource/test_azure.py b/tests/unittests/test_datasource/test_azure.py
index 42f49e0..b17f389 100644
--- a/tests/unittests/test_datasource/test_azure.py
+++ b/tests/unittests/test_datasource/test_azure.py
@@ -263,7 +263,8 @@ fdescfs /dev/fd fdescfs rw 0 0
263 {}, distro=None, paths=self.paths)263 {}, distro=None, paths=self.paths)
264 self.assertFalse(dsrc.get_data())264 self.assertFalse(dsrc.get_data())
265 self.assertEqual(265 self.assertEqual(
266 "Non-Azure DMI asset tag '{0}' discovered.\n".format(nonazure_tag),266 "DEBUG: Non-Azure DMI asset tag '{0}' discovered.\n".format(
267 nonazure_tag),
267 self.logs.getvalue())268 self.logs.getvalue())
268269
269 def test_basic_seed_dir(self):270 def test_basic_seed_dir(self):
diff --git a/tests/unittests/test_handler/test_handler_ntp.py b/tests/unittests/test_handler/test_handler_ntp.py
index 3a9f7f7..c4299d9 100644
--- a/tests/unittests/test_handler/test_handler_ntp.py
+++ b/tests/unittests/test_handler/test_handler_ntp.py
@@ -216,7 +216,8 @@ class TestNtp(FilesystemMockingTestCase):
216 """When no ntp section is defined handler logs a warning and noops."""216 """When no ntp section is defined handler logs a warning and noops."""
217 cc_ntp.handle('cc_ntp', {}, None, None, [])217 cc_ntp.handle('cc_ntp', {}, None, None, [])
218 self.assertEqual(218 self.assertEqual(
219 'Skipping module named cc_ntp, not present or disabled by cfg\n',219 'DEBUG: Skipping module named cc_ntp, '
220 'not present or disabled by cfg\n',
220 self.logs.getvalue())221 self.logs.getvalue())
221222
222 def test_ntp_handler_schema_validation_allows_empty_ntp_config(self):223 def test_ntp_handler_schema_validation_allows_empty_ntp_config(self):
diff --git a/tests/unittests/test_handler/test_handler_write_files.py b/tests/unittests/test_handler/test_handler_write_files.py
index fb252d1..88a4742 100644
--- a/tests/unittests/test_handler/test_handler_write_files.py
+++ b/tests/unittests/test_handler/test_handler_write_files.py
@@ -1,10 +1,10 @@
1# This file is part of cloud-init. See LICENSE file for license information.1# This file is part of cloud-init. See LICENSE file for license information.
22
3from cloudinit.config.cc_write_files import write_files3from cloudinit.config.cc_write_files import write_files, decode_perms
4from cloudinit import log as logging4from cloudinit import log as logging
5from cloudinit import util5from cloudinit import util
66
7from ..helpers import FilesystemMockingTestCase7from ..helpers import CiTestCase, FilesystemMockingTestCase
88
9import base649import base64
10import gzip10import gzip
@@ -98,6 +98,33 @@ class TestWriteFiles(FilesystemMockingTestCase):
98 self.assertEqual(len(expected), flen_expected)98 self.assertEqual(len(expected), flen_expected)
9999
100100
101class TestDecodePerms(CiTestCase):
102
103 with_logs = True
104
105 def test_none_returns_default(self):
106 """If None is passed as perms, then default should be returned."""
107 default = object()
108 found = decode_perms(None, default, self.logger)
109 self.assertEqual(default, found)
110
111 def test_integer(self):
112 """A valid integer should return itself."""
113 found = decode_perms(0o755, None, self.logger)
114 self.assertEqual(0o755, found)
115
116 def test_valid_octal_string(self):
117 """A string should be read as octal."""
118 found = decode_perms("644", None, self.logger)
119 self.assertEqual(0o644, found)
120
121 def test_invalid_octal_string_returns_default_and_warns(self):
122 """A string with invalid octal should warn and return default."""
123 found = decode_perms("999", None, self.logger)
124 self.assertIsNone(found)
125 self.assertIn("WARNING: Undecodable", self.logs.getvalue())
126
127
101def _gzip_bytes(data):128def _gzip_bytes(data):
102 buf = six.BytesIO()129 buf = six.BytesIO()
103 fp = None130 fp = None

Subscribers

People subscribed via source and target branches