Merge lp:~gholt/swift/logroutes into lp:~hudson-openstack/swift/trunk
- logroutes
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | David Goetz |
Approved revision: | 213 |
Merged at revision: | 214 |
Proposed branch: | lp:~gholt/swift/logroutes |
Merge into: | lp:~hudson-openstack/swift/trunk |
Diff against target: |
501 lines (+48/-44) 33 files modified
bin/swift-account-stats-logger (+1/-1) bin/swift-drive-audit (+2/-1) bin/swift-log-uploader (+1/-1) swift/account/auditor.py (+1/-1) swift/account/reaper.py (+1/-1) swift/account/server.py (+1/-1) swift/auth/server.py (+1/-1) swift/common/daemon.py (+2/-2) swift/common/db.py (+2/-2) swift/common/db_replicator.py (+1/-1) swift/common/middleware/catch_errors.py (+1/-5) swift/common/middleware/cname_lookup.py (+1/-1) swift/common/middleware/ratelimit.py (+1/-1) swift/common/middleware/swauth.py (+1/-1) swift/common/utils.py (+2/-0) swift/common/wsgi.py (+1/-1) swift/container/auditor.py (+1/-1) swift/container/server.py (+1/-1) swift/container/updater.py (+1/-1) swift/obj/auditor.py (+1/-1) swift/obj/replicator.py (+1/-1) swift/obj/server.py (+1/-1) swift/obj/updater.py (+1/-1) swift/proxy/server.py (+1/-1) swift/stats/access_processor.py (+1/-1) swift/stats/account_stats.py (+2/-1) swift/stats/log_processor.py (+2/-2) swift/stats/log_uploader.py (+3/-2) swift/stats/stats_processor.py (+1/-1) test/unit/auth/test_server.py (+2/-2) test/unit/common/test_daemon.py (+2/-2) test/unit/common/test_utils.py (+5/-3) test/unit/obj/test_auditor.py (+2/-1) |
To merge this branch: | bzr merge lp:~gholt/swift/logroutes |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
clayg | Approve | ||
Review via email: mp+48388@code.launchpad.net |
Commit message
logging: use routes to separate logging configurations
Description of the change
logging: use routes to separate logging configurations
Specifically, this fixes using different log_level settings even if the same log_name is used.
clayg (clay-gerrard) wrote : | # |
gholt (gholt) wrote : | # |
No, all that was intended. This separates the naming from the routing, then tries to get the naming defaults configured back to what they originally were, or what I thought was intended. I just missed some naming on items apparently.
I'll get the drive-audit name fixed. With the log processing stuff, I got lost pretty quick on what was intended to be named what and meant to tap you and John on that (but obviously forgot). Most everything in the system logs with no "swift-" prefix, but some of the log stuff was previously.
gholt (gholt) wrote : | # |
Fixed the drive-audit log name.
Maybe this comment doesn't belong here, but it's logging related so maybe it does, I dunno. :)
The log processing stuff is a bit confusing to me since it doesn't follow the model the rest of the code does. i.e. binary name ~= section name == default log name. It doesn't even seem to have consistency on it's own, really.
From trunk it seems to be:
The bin/swift-
The bin/swift-
The bin/swift-
So, perhaps I can be forgiven for not knowing what to do with these during this logging refactor? I really don't want to mimic this behavior in this branch, unless that's really, really what's needed.
gholt (gholt) wrote : | # |
Desnarkified by clayg and ready for re-review!
Preview Diff
1 | === modified file 'bin/swift-account-stats-logger' | |||
2 | --- bin/swift-account-stats-logger 2011-01-04 23:34:43 +0000 | |||
3 | +++ bin/swift-account-stats-logger 2011-02-10 23:12:47 +0000 | |||
4 | @@ -23,4 +23,4 @@ | |||
5 | 23 | # currently AccountStat only supports run_once | 23 | # currently AccountStat only supports run_once |
6 | 24 | options['once'] = True | 24 | options['once'] = True |
7 | 25 | run_daemon(AccountStat, conf_file, section_name='log-processor-stats', | 25 | run_daemon(AccountStat, conf_file, section_name='log-processor-stats', |
9 | 26 | **options) | 26 | log_name="account-stats", **options) |
10 | 27 | 27 | ||
11 | === modified file 'bin/swift-drive-audit' | |||
12 | --- bin/swift-drive-audit 2011-01-26 22:31:33 +0000 | |||
13 | +++ bin/swift-drive-audit 2011-02-10 23:12:47 +0000 | |||
14 | @@ -99,7 +99,8 @@ | |||
15 | 99 | device_dir = conf.get('device_dir', '/srv/node') | 99 | device_dir = conf.get('device_dir', '/srv/node') |
16 | 100 | minutes = int(conf.get('minutes', 60)) | 100 | minutes = int(conf.get('minutes', 60)) |
17 | 101 | error_limit = int(conf.get('error_limit', 1)) | 101 | error_limit = int(conf.get('error_limit', 1)) |
19 | 102 | logger = get_logger(conf, 'drive-audit') | 102 | conf['log_name'] = conf.get('log_name', 'drive-audit') |
20 | 103 | logger = get_logger(conf, log_route='drive-audit') | ||
21 | 103 | devices = get_devices(device_dir, logger) | 104 | devices = get_devices(device_dir, logger) |
22 | 104 | logger.debug("Devices found: %s" % str(devices)) | 105 | logger.debug("Devices found: %s" % str(devices)) |
23 | 105 | if not devices: | 106 | if not devices: |
24 | 106 | 107 | ||
25 | === modified file 'bin/swift-log-uploader' | |||
26 | --- bin/swift-log-uploader 2011-01-04 23:34:43 +0000 | |||
27 | +++ bin/swift-log-uploader 2011-02-10 23:12:47 +0000 | |||
28 | @@ -34,7 +34,7 @@ | |||
29 | 34 | uploader_conf.update(plugin_conf) | 34 | uploader_conf.update(plugin_conf) |
30 | 35 | 35 | ||
31 | 36 | # pre-configure logger | 36 | # pre-configure logger |
33 | 37 | logger = utils.get_logger(uploader_conf, plugin, | 37 | logger = utils.get_logger(uploader_conf, log_route='log-uploader', |
34 | 38 | log_to_console=options.get('verbose', False)) | 38 | log_to_console=options.get('verbose', False)) |
35 | 39 | # currently LogUploader only supports run_once | 39 | # currently LogUploader only supports run_once |
36 | 40 | options['once'] = True | 40 | options['once'] = True |
37 | 41 | 41 | ||
38 | === modified file 'swift/account/auditor.py' | |||
39 | --- swift/account/auditor.py 2011-01-04 23:34:43 +0000 | |||
40 | +++ swift/account/auditor.py 2011-02-10 23:12:47 +0000 | |||
41 | @@ -28,7 +28,7 @@ | |||
42 | 28 | 28 | ||
43 | 29 | def __init__(self, conf): | 29 | def __init__(self, conf): |
44 | 30 | self.conf = conf | 30 | self.conf = conf |
46 | 31 | self.logger = get_logger(conf, 'account-auditor') | 31 | self.logger = get_logger(conf, log_route='account-auditor') |
47 | 32 | self.devices = conf.get('devices', '/srv/node') | 32 | self.devices = conf.get('devices', '/srv/node') |
48 | 33 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 33 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
49 | 34 | ('true', 't', '1', 'on', 'yes', 'y') | 34 | ('true', 't', '1', 'on', 'yes', 'y') |
50 | 35 | 35 | ||
51 | === modified file 'swift/account/reaper.py' | |||
52 | --- swift/account/reaper.py 2011-01-19 23:21:57 +0000 | |||
53 | +++ swift/account/reaper.py 2011-02-10 23:12:47 +0000 | |||
54 | @@ -53,7 +53,7 @@ | |||
55 | 53 | 53 | ||
56 | 54 | def __init__(self, conf): | 54 | def __init__(self, conf): |
57 | 55 | self.conf = conf | 55 | self.conf = conf |
59 | 56 | self.logger = get_logger(conf) | 56 | self.logger = get_logger(conf, log_route='account-reaper') |
60 | 57 | self.devices = conf.get('devices', '/srv/node') | 57 | self.devices = conf.get('devices', '/srv/node') |
61 | 58 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 58 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
62 | 59 | ('true', 't', '1', 'on', 'yes', 'y') | 59 | ('true', 't', '1', 'on', 'yes', 'y') |
63 | 60 | 60 | ||
64 | === modified file 'swift/account/server.py' | |||
65 | --- swift/account/server.py 2011-01-29 19:26:06 +0000 | |||
66 | +++ swift/account/server.py 2011-02-10 23:12:47 +0000 | |||
67 | @@ -42,7 +42,7 @@ | |||
68 | 42 | """WSGI controller for the account server.""" | 42 | """WSGI controller for the account server.""" |
69 | 43 | 43 | ||
70 | 44 | def __init__(self, conf): | 44 | def __init__(self, conf): |
72 | 45 | self.logger = get_logger(conf) | 45 | self.logger = get_logger(conf, log_route='account-server') |
73 | 46 | self.root = conf.get('devices', '/srv/node') | 46 | self.root = conf.get('devices', '/srv/node') |
74 | 47 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 47 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
75 | 48 | ('true', 't', '1', 'on', 'yes', 'y') | 48 | ('true', 't', '1', 'on', 'yes', 'y') |
76 | 49 | 49 | ||
77 | === modified file 'swift/auth/server.py' | |||
78 | --- swift/auth/server.py 2011-01-26 22:38:13 +0000 | |||
79 | +++ swift/auth/server.py 2011-02-10 23:12:47 +0000 | |||
80 | @@ -90,7 +90,7 @@ | |||
81 | 90 | """ | 90 | """ |
82 | 91 | 91 | ||
83 | 92 | def __init__(self, conf): | 92 | def __init__(self, conf): |
85 | 93 | self.logger = get_logger(conf) | 93 | self.logger = get_logger(conf, log_route='auth-server') |
86 | 94 | self.super_admin_key = conf.get('super_admin_key') | 94 | self.super_admin_key = conf.get('super_admin_key') |
87 | 95 | if not self.super_admin_key: | 95 | if not self.super_admin_key: |
88 | 96 | msg = _('No super_admin_key set in conf file! Exiting.') | 96 | msg = _('No super_admin_key set in conf file! Exiting.') |
89 | 97 | 97 | ||
90 | === modified file 'swift/common/daemon.py' | |||
91 | --- swift/common/daemon.py 2011-01-04 23:34:43 +0000 | |||
92 | +++ swift/common/daemon.py 2011-02-10 23:12:47 +0000 | |||
93 | @@ -26,7 +26,7 @@ | |||
94 | 26 | 26 | ||
95 | 27 | def __init__(self, conf): | 27 | def __init__(self, conf): |
96 | 28 | self.conf = conf | 28 | self.conf = conf |
98 | 29 | self.logger = utils.get_logger(conf, 'swift-daemon') | 29 | self.logger = utils.get_logger(conf, log_route='daemon') |
99 | 30 | 30 | ||
100 | 31 | def run_once(self): | 31 | def run_once(self): |
101 | 32 | """Override this to run the script once""" | 32 | """Override this to run the script once""" |
102 | @@ -84,7 +84,7 @@ | |||
103 | 84 | logger = kwargs.pop('logger') | 84 | logger = kwargs.pop('logger') |
104 | 85 | else: | 85 | else: |
105 | 86 | logger = utils.get_logger(conf, conf.get('log_name', section_name), | 86 | logger = utils.get_logger(conf, conf.get('log_name', section_name), |
107 | 87 | log_to_console=kwargs.pop('verbose', False)) | 87 | log_to_console=kwargs.pop('verbose', False), log_route=section_name) |
108 | 88 | try: | 88 | try: |
109 | 89 | klass(conf).run(once=once, **kwargs) | 89 | klass(conf).run(once=once, **kwargs) |
110 | 90 | except KeyboardInterrupt: | 90 | except KeyboardInterrupt: |
111 | 91 | 91 | ||
112 | === modified file 'swift/common/db.py' | |||
113 | --- swift/common/db.py 2011-02-04 19:50:30 +0000 | |||
114 | +++ swift/common/db.py 2011-02-10 23:12:47 +0000 | |||
115 | @@ -287,7 +287,7 @@ | |||
116 | 287 | self.conn = None | 287 | self.conn = None |
117 | 288 | orig_isolation_level = conn.isolation_level | 288 | orig_isolation_level = conn.isolation_level |
118 | 289 | conn.isolation_level = None | 289 | conn.isolation_level = None |
120 | 290 | conn.execute('PRAGMA journal_mode = DELETE') # remove any journal files | 290 | conn.execute('PRAGMA journal_mode = DELETE') # remove journal files |
121 | 291 | conn.execute('BEGIN IMMEDIATE') | 291 | conn.execute('BEGIN IMMEDIATE') |
122 | 292 | try: | 292 | try: |
123 | 293 | yield True | 293 | yield True |
124 | @@ -295,7 +295,7 @@ | |||
125 | 295 | pass | 295 | pass |
126 | 296 | try: | 296 | try: |
127 | 297 | conn.execute('ROLLBACK') | 297 | conn.execute('ROLLBACK') |
129 | 298 | conn.execute('PRAGMA journal_mode = WAL') # back to WAL mode | 298 | conn.execute('PRAGMA journal_mode = WAL') # back to WAL mode |
130 | 299 | conn.isolation_level = orig_isolation_level | 299 | conn.isolation_level = orig_isolation_level |
131 | 300 | self.conn = conn | 300 | self.conn = conn |
132 | 301 | except Exception: | 301 | except Exception: |
133 | 302 | 302 | ||
134 | === modified file 'swift/common/db_replicator.py' | |||
135 | --- swift/common/db_replicator.py 2011-01-29 19:26:06 +0000 | |||
136 | +++ swift/common/db_replicator.py 2011-02-10 23:12:47 +0000 | |||
137 | @@ -92,7 +92,7 @@ | |||
138 | 92 | 92 | ||
139 | 93 | def __init__(self, conf): | 93 | def __init__(self, conf): |
140 | 94 | self.conf = conf | 94 | self.conf = conf |
142 | 95 | self.logger = get_logger(conf) | 95 | self.logger = get_logger(conf, log_route='replicator') |
143 | 96 | self.root = conf.get('devices', '/srv/node') | 96 | self.root = conf.get('devices', '/srv/node') |
144 | 97 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 97 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
145 | 98 | ('true', 't', '1', 'on', 'yes', 'y') | 98 | ('true', 't', '1', 'on', 'yes', 'y') |
146 | 99 | 99 | ||
147 | === modified file 'swift/common/middleware/catch_errors.py' | |||
148 | --- swift/common/middleware/catch_errors.py 2011-01-07 21:17:29 +0000 | |||
149 | +++ swift/common/middleware/catch_errors.py 2011-02-10 23:12:47 +0000 | |||
150 | @@ -26,11 +26,7 @@ | |||
151 | 26 | 26 | ||
152 | 27 | def __init__(self, app, conf): | 27 | def __init__(self, app, conf): |
153 | 28 | self.app = app | 28 | self.app = app |
159 | 29 | # if the application already has a logger we should use that one | 29 | self.logger = get_logger(conf, log_route='catch-errors') |
155 | 30 | self.logger = getattr(app, 'logger', None) | ||
156 | 31 | if not self.logger: | ||
157 | 32 | # and only call get_logger if we have to | ||
158 | 33 | self.logger = get_logger(conf) | ||
160 | 34 | 30 | ||
161 | 35 | def __call__(self, env, start_response): | 31 | def __call__(self, env, start_response): |
162 | 36 | try: | 32 | try: |
163 | 37 | 33 | ||
164 | === modified file 'swift/common/middleware/cname_lookup.py' | |||
165 | --- swift/common/middleware/cname_lookup.py 2011-01-20 20:15:05 +0000 | |||
166 | +++ swift/common/middleware/cname_lookup.py 2011-02-10 23:12:47 +0000 | |||
167 | @@ -53,7 +53,7 @@ | |||
168 | 53 | self.storage_domain = '.' + self.storage_domain | 53 | self.storage_domain = '.' + self.storage_domain |
169 | 54 | self.lookup_depth = int(conf.get('lookup_depth', '1')) | 54 | self.lookup_depth = int(conf.get('lookup_depth', '1')) |
170 | 55 | self.memcache = None | 55 | self.memcache = None |
172 | 56 | self.logger = get_logger(conf) | 56 | self.logger = get_logger(conf, log_route='cname-lookup') |
173 | 57 | 57 | ||
174 | 58 | def __call__(self, env, start_response): | 58 | def __call__(self, env, start_response): |
175 | 59 | if not self.storage_domain: | 59 | if not self.storage_domain: |
176 | 60 | 60 | ||
177 | === modified file 'swift/common/middleware/ratelimit.py' | |||
178 | --- swift/common/middleware/ratelimit.py 2011-01-22 00:28:58 +0000 | |||
179 | +++ swift/common/middleware/ratelimit.py 2011-02-10 23:12:47 +0000 | |||
180 | @@ -39,7 +39,7 @@ | |||
181 | 39 | if logger: | 39 | if logger: |
182 | 40 | self.logger = logger | 40 | self.logger = logger |
183 | 41 | else: | 41 | else: |
185 | 42 | self.logger = get_logger(conf) | 42 | self.logger = get_logger(conf, log_route='ratelimit') |
186 | 43 | self.account_ratelimit = float(conf.get('account_ratelimit', 0)) | 43 | self.account_ratelimit = float(conf.get('account_ratelimit', 0)) |
187 | 44 | self.max_sleep_time_seconds = \ | 44 | self.max_sleep_time_seconds = \ |
188 | 45 | float(conf.get('max_sleep_time_seconds', 60)) | 45 | float(conf.get('max_sleep_time_seconds', 60)) |
189 | 46 | 46 | ||
190 | === modified file 'swift/common/middleware/swauth.py' | |||
191 | --- swift/common/middleware/swauth.py 2011-02-02 18:23:01 +0000 | |||
192 | +++ swift/common/middleware/swauth.py 2011-02-10 23:12:47 +0000 | |||
193 | @@ -51,7 +51,7 @@ | |||
194 | 51 | def __init__(self, app, conf): | 51 | def __init__(self, app, conf): |
195 | 52 | self.app = app | 52 | self.app = app |
196 | 53 | self.conf = conf | 53 | self.conf = conf |
198 | 54 | self.logger = get_logger(conf) | 54 | self.logger = get_logger(conf, log_route='swauth') |
199 | 55 | self.log_headers = conf.get('log_headers') == 'True' | 55 | self.log_headers = conf.get('log_headers') == 'True' |
200 | 56 | self.reseller_prefix = conf.get('reseller_prefix', 'AUTH').strip() | 56 | self.reseller_prefix = conf.get('reseller_prefix', 'AUTH').strip() |
201 | 57 | if self.reseller_prefix and self.reseller_prefix[-1] != '_': | 57 | if self.reseller_prefix and self.reseller_prefix[-1] != '_': |
202 | 58 | 58 | ||
203 | === modified file 'swift/common/utils.py' | |||
204 | --- swift/common/utils.py 2011-02-10 21:23:59 +0000 | |||
205 | +++ swift/common/utils.py 2011-02-10 23:12:47 +0000 | |||
206 | @@ -390,6 +390,8 @@ | |||
207 | 390 | :param conf: Configuration dict to read settings from | 390 | :param conf: Configuration dict to read settings from |
208 | 391 | :param name: Name of the logger | 391 | :param name: Name of the logger |
209 | 392 | :param log_to_console: Add handler which writes to console on stderr | 392 | :param log_to_console: Add handler which writes to console on stderr |
210 | 393 | :param log_route: Route for the logging, not emitted to the log, just used | ||
211 | 394 | to separate logging configurations | ||
212 | 393 | :param fmt: Override log format | 395 | :param fmt: Override log format |
213 | 394 | """ | 396 | """ |
214 | 395 | if not conf: | 397 | if not conf: |
215 | 396 | 398 | ||
216 | === modified file 'swift/common/wsgi.py' | |||
217 | --- swift/common/wsgi.py 2011-02-05 21:38:49 +0000 | |||
218 | +++ swift/common/wsgi.py 2011-02-10 23:12:47 +0000 | |||
219 | @@ -113,7 +113,7 @@ | |||
220 | 113 | logger = kwargs.pop('logger') | 113 | logger = kwargs.pop('logger') |
221 | 114 | else: | 114 | else: |
222 | 115 | logger = get_logger(conf, log_name, | 115 | logger = get_logger(conf, log_name, |
224 | 116 | log_to_console=kwargs.pop('verbose', False)) | 116 | log_to_console=kwargs.pop('verbose', False), log_route='wsgi') |
225 | 117 | 117 | ||
226 | 118 | # redirect errors to logger and close stdio | 118 | # redirect errors to logger and close stdio |
227 | 119 | capture_stdio(logger) | 119 | capture_stdio(logger) |
228 | 120 | 120 | ||
229 | === modified file 'swift/container/auditor.py' | |||
230 | --- swift/container/auditor.py 2011-01-04 23:34:43 +0000 | |||
231 | +++ swift/container/auditor.py 2011-02-10 23:12:47 +0000 | |||
232 | @@ -28,7 +28,7 @@ | |||
233 | 28 | 28 | ||
234 | 29 | def __init__(self, conf): | 29 | def __init__(self, conf): |
235 | 30 | self.conf = conf | 30 | self.conf = conf |
237 | 31 | self.logger = get_logger(conf, 'container-auditor') | 31 | self.logger = get_logger(conf, log_route='container-auditor') |
238 | 32 | self.devices = conf.get('devices', '/srv/node') | 32 | self.devices = conf.get('devices', '/srv/node') |
239 | 33 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 33 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
240 | 34 | ('true', 't', '1', 'on', 'yes', 'y') | 34 | ('true', 't', '1', 'on', 'yes', 'y') |
241 | 35 | 35 | ||
242 | === modified file 'swift/container/server.py' | |||
243 | --- swift/container/server.py 2011-01-29 19:26:06 +0000 | |||
244 | +++ swift/container/server.py 2011-02-10 23:12:47 +0000 | |||
245 | @@ -49,7 +49,7 @@ | |||
246 | 49 | save_headers = ['x-container-read', 'x-container-write'] | 49 | save_headers = ['x-container-read', 'x-container-write'] |
247 | 50 | 50 | ||
248 | 51 | def __init__(self, conf): | 51 | def __init__(self, conf): |
250 | 52 | self.logger = get_logger(conf) | 52 | self.logger = get_logger(conf, log_route='container-server') |
251 | 53 | self.root = conf.get('devices', '/srv/node/') | 53 | self.root = conf.get('devices', '/srv/node/') |
252 | 54 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 54 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
253 | 55 | ('true', 't', '1', 'on', 'yes', 'y') | 55 | ('true', 't', '1', 'on', 'yes', 'y') |
254 | 56 | 56 | ||
255 | === modified file 'swift/container/updater.py' | |||
256 | --- swift/container/updater.py 2011-01-26 22:38:13 +0000 | |||
257 | +++ swift/container/updater.py 2011-02-10 23:12:47 +0000 | |||
258 | @@ -37,7 +37,7 @@ | |||
259 | 37 | 37 | ||
260 | 38 | def __init__(self, conf): | 38 | def __init__(self, conf): |
261 | 39 | self.conf = conf | 39 | self.conf = conf |
263 | 40 | self.logger = get_logger(conf, 'container-updater') | 40 | self.logger = get_logger(conf, log_route='container-updater') |
264 | 41 | self.devices = conf.get('devices', '/srv/node') | 41 | self.devices = conf.get('devices', '/srv/node') |
265 | 42 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 42 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
266 | 43 | ('true', 't', '1', 'on', 'yes', 'y') | 43 | ('true', 't', '1', 'on', 'yes', 'y') |
267 | 44 | 44 | ||
268 | === modified file 'swift/obj/auditor.py' | |||
269 | --- swift/obj/auditor.py 2011-01-21 01:05:44 +0000 | |||
270 | +++ swift/obj/auditor.py 2011-02-10 23:12:47 +0000 | |||
271 | @@ -31,7 +31,7 @@ | |||
272 | 31 | 31 | ||
273 | 32 | def __init__(self, conf): | 32 | def __init__(self, conf): |
274 | 33 | self.conf = conf | 33 | self.conf = conf |
276 | 34 | self.logger = get_logger(conf, 'object-auditor') | 34 | self.logger = get_logger(conf, log_route='object-auditor') |
277 | 35 | self.devices = conf.get('devices', '/srv/node') | 35 | self.devices = conf.get('devices', '/srv/node') |
278 | 36 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 36 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
279 | 37 | ('true', 't', '1', 'on', 'yes', 'y') | 37 | ('true', 't', '1', 'on', 'yes', 'y') |
280 | 38 | 38 | ||
281 | === modified file 'swift/obj/replicator.py' | |||
282 | --- swift/obj/replicator.py 2011-01-27 22:42:36 +0000 | |||
283 | +++ swift/obj/replicator.py 2011-02-10 23:12:47 +0000 | |||
284 | @@ -207,7 +207,7 @@ | |||
285 | 207 | :param logger: logging object | 207 | :param logger: logging object |
286 | 208 | """ | 208 | """ |
287 | 209 | self.conf = conf | 209 | self.conf = conf |
289 | 210 | self.logger = get_logger(conf, 'object-replicator') | 210 | self.logger = get_logger(conf, log_route='object-replicator') |
290 | 211 | self.devices_dir = conf.get('devices', '/srv/node') | 211 | self.devices_dir = conf.get('devices', '/srv/node') |
291 | 212 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 212 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
292 | 213 | ('true', 't', '1', 'on', 'yes', 'y') | 213 | ('true', 't', '1', 'on', 'yes', 'y') |
293 | 214 | 214 | ||
294 | === modified file 'swift/obj/server.py' | |||
295 | --- swift/obj/server.py 2011-01-26 22:38:13 +0000 | |||
296 | +++ swift/obj/server.py 2011-02-10 23:12:47 +0000 | |||
297 | @@ -266,7 +266,7 @@ | |||
298 | 266 | <source-dir>/etc/object-server.conf-sample or | 266 | <source-dir>/etc/object-server.conf-sample or |
299 | 267 | /etc/swift/object-server.conf-sample. | 267 | /etc/swift/object-server.conf-sample. |
300 | 268 | """ | 268 | """ |
302 | 269 | self.logger = get_logger(conf) | 269 | self.logger = get_logger(conf, log_route='object-server') |
303 | 270 | self.devices = conf.get('devices', '/srv/node/') | 270 | self.devices = conf.get('devices', '/srv/node/') |
304 | 271 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 271 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
305 | 272 | ('true', 't', '1', 'on', 'yes', 'y') | 272 | ('true', 't', '1', 'on', 'yes', 'y') |
306 | 273 | 273 | ||
307 | === modified file 'swift/obj/updater.py' | |||
308 | --- swift/obj/updater.py 2011-01-26 22:31:33 +0000 | |||
309 | +++ swift/obj/updater.py 2011-02-10 23:12:47 +0000 | |||
310 | @@ -35,7 +35,7 @@ | |||
311 | 35 | 35 | ||
312 | 36 | def __init__(self, conf): | 36 | def __init__(self, conf): |
313 | 37 | self.conf = conf | 37 | self.conf = conf |
315 | 38 | self.logger = get_logger(conf, 'object-updater') | 38 | self.logger = get_logger(conf, log_route='object-updater') |
316 | 39 | self.devices = conf.get('devices', '/srv/node') | 39 | self.devices = conf.get('devices', '/srv/node') |
317 | 40 | self.mount_check = conf.get('mount_check', 'true').lower() in \ | 40 | self.mount_check = conf.get('mount_check', 'true').lower() in \ |
318 | 41 | ('true', 't', '1', 'on', 'yes', 'y') | 41 | ('true', 't', '1', 'on', 'yes', 'y') |
319 | 42 | 42 | ||
320 | === modified file 'swift/proxy/server.py' | |||
321 | --- swift/proxy/server.py 2011-02-10 20:59:52 +0000 | |||
322 | +++ swift/proxy/server.py 2011-02-10 23:12:47 +0000 | |||
323 | @@ -1609,7 +1609,7 @@ | |||
324 | 1609 | if conf is None: | 1609 | if conf is None: |
325 | 1610 | conf = {} | 1610 | conf = {} |
326 | 1611 | if logger is None: | 1611 | if logger is None: |
328 | 1612 | self.logger = get_logger(conf) | 1612 | self.logger = get_logger(conf, log_route='proxy-server') |
329 | 1613 | access_log_conf = {} | 1613 | access_log_conf = {} |
330 | 1614 | for key in ('log_facility', 'log_name', 'log_level'): | 1614 | for key in ('log_facility', 'log_name', 'log_level'): |
331 | 1615 | value = conf.get('access_' + key, conf.get(key, None)) | 1615 | value = conf.get('access_' + key, conf.get(key, None)) |
332 | 1616 | 1616 | ||
333 | === modified file 'swift/stats/access_processor.py' | |||
334 | --- swift/stats/access_processor.py 2011-01-17 17:07:58 +0000 | |||
335 | +++ swift/stats/access_processor.py 2011-02-10 23:12:47 +0000 | |||
336 | @@ -34,7 +34,7 @@ | |||
337 | 34 | conf.get('service_ips', '').split(',')\ | 34 | conf.get('service_ips', '').split(',')\ |
338 | 35 | if x.strip()] | 35 | if x.strip()] |
339 | 36 | self.warn_percent = float(conf.get('warn_percent', '0.8')) | 36 | self.warn_percent = float(conf.get('warn_percent', '0.8')) |
341 | 37 | self.logger = get_logger(conf) | 37 | self.logger = get_logger(conf, log_route='access-processor') |
342 | 38 | 38 | ||
343 | 39 | def log_line_parser(self, raw_log): | 39 | def log_line_parser(self, raw_log): |
344 | 40 | '''given a raw access log line, return a dict of the good parts''' | 40 | '''given a raw access log line, return a dict of the good parts''' |
345 | 41 | 41 | ||
346 | === modified file 'swift/stats/account_stats.py' | |||
347 | --- swift/stats/account_stats.py 2011-01-30 14:59:35 +0000 | |||
348 | +++ swift/stats/account_stats.py 2011-02-10 23:12:47 +0000 | |||
349 | @@ -48,7 +48,8 @@ | |||
350 | 48 | self.devices = server_conf.get('devices', '/srv/node') | 48 | self.devices = server_conf.get('devices', '/srv/node') |
351 | 49 | self.mount_check = server_conf.get('mount_check', 'true').lower() in \ | 49 | self.mount_check = server_conf.get('mount_check', 'true').lower() in \ |
352 | 50 | ('true', 't', '1', 'on', 'yes', 'y') | 50 | ('true', 't', '1', 'on', 'yes', 'y') |
354 | 51 | self.logger = get_logger(stats_conf, 'swift-account-stats-logger') | 51 | self.logger = \ |
355 | 52 | get_logger(stats_conf, log_route='account-stats') | ||
356 | 52 | 53 | ||
357 | 53 | def run_once(self): | 54 | def run_once(self): |
358 | 54 | self.logger.info(_("Gathering account stats")) | 55 | self.logger.info(_("Gathering account stats")) |
359 | 55 | 56 | ||
360 | === modified file 'swift/stats/log_processor.py' | |||
361 | --- swift/stats/log_processor.py 2011-01-26 22:38:13 +0000 | |||
362 | +++ swift/stats/log_processor.py 2011-02-10 23:12:47 +0000 | |||
363 | @@ -40,7 +40,7 @@ | |||
364 | 40 | 40 | ||
365 | 41 | def __init__(self, conf, logger): | 41 | def __init__(self, conf, logger): |
366 | 42 | if isinstance(logger, tuple): | 42 | if isinstance(logger, tuple): |
368 | 43 | self.logger = get_logger(*logger) | 43 | self.logger = get_logger(*logger, log_route='log-processor') |
369 | 44 | else: | 44 | else: |
370 | 45 | self.logger = logger | 45 | self.logger = logger |
371 | 46 | 46 | ||
372 | @@ -226,7 +226,7 @@ | |||
373 | 226 | c = conf.get('log-processor') | 226 | c = conf.get('log-processor') |
374 | 227 | super(LogProcessorDaemon, self).__init__(c) | 227 | super(LogProcessorDaemon, self).__init__(c) |
375 | 228 | self.total_conf = conf | 228 | self.total_conf = conf |
377 | 229 | self.logger = get_logger(c) | 229 | self.logger = get_logger(c, log_route='log-processor') |
378 | 230 | self.log_processor = LogProcessor(conf, self.logger) | 230 | self.log_processor = LogProcessor(conf, self.logger) |
379 | 231 | self.lookback_hours = int(c.get('lookback_hours', '120')) | 231 | self.lookback_hours = int(c.get('lookback_hours', '120')) |
380 | 232 | self.lookback_window = int(c.get('lookback_window', | 232 | self.lookback_window = int(c.get('lookback_window', |
381 | 233 | 233 | ||
382 | === modified file 'swift/stats/log_uploader.py' | |||
383 | --- swift/stats/log_uploader.py 2011-01-14 08:45:39 +0000 | |||
384 | +++ swift/stats/log_uploader.py 2011-02-10 23:12:47 +0000 | |||
385 | @@ -64,8 +64,9 @@ | |||
386 | 64 | self.container_name = container_name | 64 | self.container_name = container_name |
387 | 65 | self.filename_format = source_filename_format | 65 | self.filename_format = source_filename_format |
388 | 66 | self.internal_proxy = InternalProxy(proxy_server_conf) | 66 | self.internal_proxy = InternalProxy(proxy_server_conf) |
391 | 67 | log_name = 'swift-log-uploader-%s' % plugin_name | 67 | log_name = '%s-log-uploader' % plugin_name |
392 | 68 | self.logger = utils.get_logger(uploader_conf, plugin_name) | 68 | self.logger = utils.get_logger(uploader_conf, log_name, |
393 | 69 | log_route=plugin_name) | ||
394 | 69 | 70 | ||
395 | 70 | def run_once(self): | 71 | def run_once(self): |
396 | 71 | self.logger.info(_("Uploading logs")) | 72 | self.logger.info(_("Uploading logs")) |
397 | 72 | 73 | ||
398 | === modified file 'swift/stats/stats_processor.py' | |||
399 | --- swift/stats/stats_processor.py 2011-01-19 23:21:57 +0000 | |||
400 | +++ swift/stats/stats_processor.py 2011-02-10 23:12:47 +0000 | |||
401 | @@ -20,7 +20,7 @@ | |||
402 | 20 | """Transform account storage stat logs""" | 20 | """Transform account storage stat logs""" |
403 | 21 | 21 | ||
404 | 22 | def __init__(self, conf): | 22 | def __init__(self, conf): |
406 | 23 | self.logger = get_logger(conf) | 23 | self.logger = get_logger(conf, log_route='stats-processor') |
407 | 24 | 24 | ||
408 | 25 | def process(self, obj_stream, data_object_account, data_object_container, | 25 | def process(self, obj_stream, data_object_account, data_object_container, |
409 | 26 | data_object_name): | 26 | data_object_name): |
410 | 27 | 27 | ||
411 | === modified file 'test/unit/auth/test_server.py' | |||
412 | --- test/unit/auth/test_server.py 2011-01-19 23:21:57 +0000 | |||
413 | +++ test/unit/auth/test_server.py 2011-02-10 23:12:47 +0000 | |||
414 | @@ -456,7 +456,7 @@ | |||
415 | 456 | def test_basic_logging(self): | 456 | def test_basic_logging(self): |
416 | 457 | log = StringIO() | 457 | log = StringIO() |
417 | 458 | log_handler = StreamHandler(log) | 458 | log_handler = StreamHandler(log) |
419 | 459 | logger = get_logger(self.conf, 'auth') | 459 | logger = get_logger(self.conf, 'auth-server', log_route='auth-server') |
420 | 460 | logger.logger.addHandler(log_handler) | 460 | logger.logger.addHandler(log_handler) |
421 | 461 | try: | 461 | try: |
422 | 462 | auth_server.http_connect = fake_http_connect(201) | 462 | auth_server.http_connect = fake_http_connect(201) |
423 | @@ -534,7 +534,7 @@ | |||
424 | 534 | orig_Request = auth_server.Request | 534 | orig_Request = auth_server.Request |
425 | 535 | log = StringIO() | 535 | log = StringIO() |
426 | 536 | log_handler = StreamHandler(log) | 536 | log_handler = StreamHandler(log) |
428 | 537 | logger = get_logger(self.conf, 'auth') | 537 | logger = get_logger(self.conf, 'auth-server', log_route='auth-server') |
429 | 538 | logger.logger.addHandler(log_handler) | 538 | logger.logger.addHandler(log_handler) |
430 | 539 | try: | 539 | try: |
431 | 540 | auth_server.Request = request_causing_exception | 540 | auth_server.Request = request_causing_exception |
432 | 541 | 541 | ||
433 | === modified file 'test/unit/common/test_daemon.py' | |||
434 | --- test/unit/common/test_daemon.py 2011-02-02 17:38:17 +0000 | |||
435 | +++ test/unit/common/test_daemon.py 2011-02-10 23:12:47 +0000 | |||
436 | @@ -28,7 +28,7 @@ | |||
437 | 28 | 28 | ||
438 | 29 | def __init__(self, conf): | 29 | def __init__(self, conf): |
439 | 30 | self.conf = conf | 30 | self.conf = conf |
441 | 31 | self.logger = utils.get_logger(None, 'server') | 31 | self.logger = utils.get_logger(None, 'server', log_route='server') |
442 | 32 | MyDaemon.forever_called = False | 32 | MyDaemon.forever_called = False |
443 | 33 | MyDaemon.once_called = False | 33 | MyDaemon.once_called = False |
444 | 34 | 34 | ||
445 | @@ -99,7 +99,7 @@ | |||
446 | 99 | sio = StringIO() | 99 | sio = StringIO() |
447 | 100 | logger = logging.getLogger('server') | 100 | logger = logging.getLogger('server') |
448 | 101 | logger.addHandler(logging.StreamHandler(sio)) | 101 | logger.addHandler(logging.StreamHandler(sio)) |
450 | 102 | logger = utils.get_logger(None, 'server') | 102 | logger = utils.get_logger(None, 'server', log_route='server') |
451 | 103 | daemon.run_daemon(MyDaemon, conf_file, logger=logger) | 103 | daemon.run_daemon(MyDaemon, conf_file, logger=logger) |
452 | 104 | self.assert_('user quit' in sio.getvalue().lower()) | 104 | self.assert_('user quit' in sio.getvalue().lower()) |
453 | 105 | 105 | ||
454 | 106 | 106 | ||
455 | === modified file 'test/unit/common/test_utils.py' | |||
456 | --- test/unit/common/test_utils.py 2011-02-10 20:59:52 +0000 | |||
457 | +++ test/unit/common/test_utils.py 2011-02-10 23:12:47 +0000 | |||
458 | @@ -303,17 +303,19 @@ | |||
459 | 303 | sio = StringIO() | 303 | sio = StringIO() |
460 | 304 | logger = logging.getLogger('server') | 304 | logger = logging.getLogger('server') |
461 | 305 | logger.addHandler(logging.StreamHandler(sio)) | 305 | logger.addHandler(logging.StreamHandler(sio)) |
463 | 306 | logger = utils.get_logger(None, 'server') | 306 | logger = utils.get_logger(None, 'server', log_route='server') |
464 | 307 | logger.warn('test1') | 307 | logger.warn('test1') |
465 | 308 | self.assertEquals(sio.getvalue(), 'test1\n') | 308 | self.assertEquals(sio.getvalue(), 'test1\n') |
466 | 309 | logger.debug('test2') | 309 | logger.debug('test2') |
467 | 310 | self.assertEquals(sio.getvalue(), 'test1\n') | 310 | self.assertEquals(sio.getvalue(), 'test1\n') |
469 | 311 | logger = utils.get_logger({'log_level': 'DEBUG'}, 'server') | 311 | logger = utils.get_logger({'log_level': 'DEBUG'}, 'server', |
470 | 312 | log_route='server') | ||
471 | 312 | logger.debug('test3') | 313 | logger.debug('test3') |
472 | 313 | self.assertEquals(sio.getvalue(), 'test1\ntest3\n') | 314 | self.assertEquals(sio.getvalue(), 'test1\ntest3\n') |
473 | 314 | # Doesn't really test that the log facility is truly being used all the | 315 | # Doesn't really test that the log facility is truly being used all the |
474 | 315 | # way to syslog; but exercises the code. | 316 | # way to syslog; but exercises the code. |
476 | 316 | logger = utils.get_logger({'log_facility': 'LOG_LOCAL3'}, 'server') | 317 | logger = utils.get_logger({'log_facility': 'LOG_LOCAL3'}, 'server', |
477 | 318 | log_route='server') | ||
478 | 317 | logger.warn('test4') | 319 | logger.warn('test4') |
479 | 318 | self.assertEquals(sio.getvalue(), | 320 | self.assertEquals(sio.getvalue(), |
480 | 319 | 'test1\ntest3\ntest4\n') | 321 | 'test1\ntest3\ntest4\n') |
481 | 320 | 322 | ||
482 | === modified file 'test/unit/obj/test_auditor.py' | |||
483 | --- test/unit/obj/test_auditor.py 2011-01-25 01:12:38 +0000 | |||
484 | +++ test/unit/obj/test_auditor.py 2011-02-10 23:12:47 +0000 | |||
485 | @@ -14,7 +14,7 @@ | |||
486 | 14 | # limitations under the License. | 14 | # limitations under the License. |
487 | 15 | 15 | ||
488 | 16 | # TODO: Tests | 16 | # TODO: Tests |
490 | 17 | from test import unit as _setup_mocks | 17 | from test import unit |
491 | 18 | import unittest | 18 | import unittest |
492 | 19 | import tempfile | 19 | import tempfile |
493 | 20 | import os | 20 | import os |
494 | @@ -57,6 +57,7 @@ | |||
495 | 57 | 57 | ||
496 | 58 | def tearDown(self): | 58 | def tearDown(self): |
497 | 59 | rmtree(os.path.dirname(self.testdir), ignore_errors=1) | 59 | rmtree(os.path.dirname(self.testdir), ignore_errors=1) |
498 | 60 | unit.xattr_data = {} | ||
499 | 60 | 61 | ||
500 | 61 | def test_object_audit_extra_data(self): | 62 | def test_object_audit_extra_data(self): |
501 | 62 | self.auditor = auditor.ObjectAuditor(self.conf) | 63 | self.auditor = auditor.ObjectAuditor(self.conf) |
Calling get_logger with a log_route kwarg instead of name as a positional argument isn't really the same thing. It may even have some side-effects that weren't intended?
e.g. stats-logger logs as "log-processor- stats" instead of "swift- account- stats-logger" or just "account-stats"
the drive-audit script logs as "swift" instead of "drive-audit"
swift-account-
I think things mostly sorta "work" because these conf dicts are coming out of readconf and run_wsgi which will both throw in a log_name key set to the section_name if there isn't something there already.
On the other hand... with this branch I can finally set a log_name for the auditors, updaters and replicators... so maybe it's a net gain?