Merge ~ajorgens/cloud-init:simpletable into cloud-init:master
- Git
- lp:~ajorgens/cloud-init
- simpletable
- Merge into master
Status: | Merged |
---|---|
Approved by: | Chad Smith |
Approved revision: | 46732a3706e83685ce494e74bac93cae2259e0c2 |
Merged at revision: | f010594beb75e146091db47b7d72d1fc1d763e98 |
Proposed branch: | ~ajorgens/cloud-init:simpletable |
Merge into: | cloud-init:master |
Diff against target: |
304 lines (+168/-16) 8 files modified
cloudinit/config/cc_ssh_authkey_fingerprints.py (+2/-2) cloudinit/netinfo.py (+4/-4) cloudinit/simpletable.py (+62/-0) cloudinit/tests/test_simpletable.py (+100/-0) packages/pkg-deps.json (+0/-3) requirements.txt (+0/-3) tools/build-on-freebsd (+0/-1) tox.ini (+0/-3) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Approve | |
Chad Smith | Approve | ||
Andrew Jorgensen (community) | Approve | ||
Review via email: mp+330525@code.launchpad.net |
Commit message
Remove prettytable dependency, introduce simpletable
The first revision of this rendered tables with less decoration but there was a desire upstream to avoid possibly breaking some parsing someone might be doing, so it has been revised to render the same as prettytable for the cases cloud-init actually uses.
Description of the change
Remove prettytable dependency, introduce simpletable
simpletable has no external dependencies and very little complexity.
Andrew Jorgensen (ajorgens) wrote : | # |
Andrew Jorgensen (ajorgens) wrote : | # |
Here's an example of netinfo output from a console log:
```
ci-info: +++++++
ci-info: Device Up Address Mask Hw-Address
ci-info: lo True 127.0.0.1 255.0.0.0 .
ci-info: eth0 True 172.31.9.66 255.255.240.0 0a:bc:31:2b:2b:7f
ci-info: +++++++
ci-info: Route Destination Gateway Genmask Interface Flags
ci-info: 0 0.0.0.0 172.31.0.1 0.0.0.0 eth0 UG
ci-info: 1 169.254.169.254 0.0.0.0 255.255.255.255 eth0 UH
ci-info: 2 172.31.0.0 0.0.0.0 255.255.240.0 eth0 U
```
Andrew Jorgensen (ajorgens) wrote : | # |
Launchpad is collapsing spaces in the above comment, so it's useless in showing what it actually looks like. Sorry.
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:69951f18fe4
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Scott Moser (smoser) wrote : | # |
Andrew, wrt useless of launchpad collapsing spaces, you can put it into a
pastebin http://
You dont remove the dep from requirements.txt here, so it probably ends
up still being a dep at least in ubuntu
I'd like to drop prettytable or at very least make it a soft dependency,
falling back to your simpletable if it wasnt there.
There is an issue.. though. We were using pretty table to render human
friendly output to the /dev/console. The expectation was that that was
human friendly, but since we were not rendering any *machine* friendly
representation of said data, its possible or likely that machines
were parsing that.
I'd like to have an actual solution for writing machine-friendly
data somewhere that we could depend on. And then i woudlnt care so much
about the user-friendly text changing.
This cleary isn't your fault, but I'm just kind of adverse to changing
something that kind of worked and possibly breaking someone without
putting somethign else in place that could be relied upon.
Andrew Jorgensen (ajorgens) wrote : | # |
We've been using this in Amazon Linux since early 2014 and not once have we heard a complaint, or even a mention of it. I think the worry that someone is parsing routing data out of console output is probably baseless, since that data is better obtained through queries against your cloud APIs if you are outside the instance, or through OS utilities or syscalls if you are inside the instance.
FWIW, the format produced by simpletable is *more* machine readable than the output of prettytable (and prettier, if you ask me and you don't remove any spaces).
Up to you. Amazon will likely continue to carry this patch.
I'll update with a removal from requirements.txt sometime. We had removed it from our RPM .spec file and I forgot it would need to be removed elsewhere.
Andrew Jorgensen (ajorgens) wrote : | # |
Okay, I went ahead and made its output look like PrettyTable (even if I'm unhappy about that), stripped out every reference to PrettyTable from the code (including requirements.txt, etc.), and wrote some unit tests that assert that the output is the same as PrettyTable for cases actually used in cloud-init.
Andrew Jorgensen (ajorgens) wrote : | # |
I think I set the state wrong, setting it back to what I actually think of this.
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:0a3b64acdd0
https:/
Executed test runs:
SUCCESS: Checkout
FAILED: Unit & Style Tests
Click here to trigger a rebuild:
https:/
Andrew Jorgensen (ajorgens) wrote : | # |
Ugh, how embarrassing. I tried to suppress some flake8 errors and did it so wrong.
Andrew Jorgensen (ajorgens) wrote : | # |
Okay, latest push should do it. 🤞
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:37383e5d99f
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Andrew Jorgensen (ajorgens) wrote : | # |
> PASSED: Continuous integration, rev:37383e5d99f
Yay!
Chad Smith (chad.smith) wrote : | # |
Approving pending discussion on inline comments. We probably won't land this until post 17.1 release because the change in dependencies might need a bit more testing on other platforms.
Chad Smith (chad.smith) : | # |
- 46732a3... by Chad Smith
-
[SimpleTable] Cleaner comments / Don't use StringIO
Changes from Chad Smith suggested on
https://code.launchpad .net/~ajorgens/ cloud-init/ +git/cloud- init/+merge/ 330525 Signed-off-by: Andrew Jorgensen <email address hidden>
Andrew Jorgensen (ajorgens) wrote : | # |
Pulled in Chad's suggestions.
Andrew Jorgensen (ajorgens) wrote : | # |
(diff comment replies)
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:46732a3706e
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Chad Smith (chad.smith) : | # |
Preview Diff
1 | diff --git a/cloudinit/config/cc_ssh_authkey_fingerprints.py b/cloudinit/config/cc_ssh_authkey_fingerprints.py |
2 | index 0066e97..35d8c57 100755 |
3 | --- a/cloudinit/config/cc_ssh_authkey_fingerprints.py |
4 | +++ b/cloudinit/config/cc_ssh_authkey_fingerprints.py |
5 | @@ -28,7 +28,7 @@ the keys can be specified, but defaults to ``md5``. |
6 | import base64 |
7 | import hashlib |
8 | |
9 | -from prettytable import PrettyTable |
10 | +from cloudinit.simpletable import SimpleTable |
11 | |
12 | from cloudinit.distros import ug_util |
13 | from cloudinit import ssh_util |
14 | @@ -74,7 +74,7 @@ def _pprint_key_entries(user, key_fn, key_entries, hash_meth='md5', |
15 | return |
16 | tbl_fields = ['Keytype', 'Fingerprint (%s)' % (hash_meth), 'Options', |
17 | 'Comment'] |
18 | - tbl = PrettyTable(tbl_fields) |
19 | + tbl = SimpleTable(tbl_fields) |
20 | for entry in key_entries: |
21 | if _is_printable_key(entry): |
22 | row = [] |
23 | diff --git a/cloudinit/netinfo.py b/cloudinit/netinfo.py |
24 | index 39c79de..8f99d99 100644 |
25 | --- a/cloudinit/netinfo.py |
26 | +++ b/cloudinit/netinfo.py |
27 | @@ -13,7 +13,7 @@ import re |
28 | from cloudinit import log as logging |
29 | from cloudinit import util |
30 | |
31 | -from prettytable import PrettyTable |
32 | +from cloudinit.simpletable import SimpleTable |
33 | |
34 | LOG = logging.getLogger() |
35 | |
36 | @@ -170,7 +170,7 @@ def netdev_pformat(): |
37 | lines.append(util.center("Net device info failed", '!', 80)) |
38 | else: |
39 | fields = ['Device', 'Up', 'Address', 'Mask', 'Scope', 'Hw-Address'] |
40 | - tbl = PrettyTable(fields) |
41 | + tbl = SimpleTable(fields) |
42 | for (dev, d) in netdev.items(): |
43 | tbl.add_row([dev, d["up"], d["addr"], d["mask"], ".", d["hwaddr"]]) |
44 | if d.get('addr6'): |
45 | @@ -194,7 +194,7 @@ def route_pformat(): |
46 | if routes.get('ipv4'): |
47 | fields_v4 = ['Route', 'Destination', 'Gateway', |
48 | 'Genmask', 'Interface', 'Flags'] |
49 | - tbl_v4 = PrettyTable(fields_v4) |
50 | + tbl_v4 = SimpleTable(fields_v4) |
51 | for (n, r) in enumerate(routes.get('ipv4')): |
52 | route_id = str(n) |
53 | tbl_v4.add_row([route_id, r['destination'], |
54 | @@ -207,7 +207,7 @@ def route_pformat(): |
55 | if routes.get('ipv6'): |
56 | fields_v6 = ['Route', 'Proto', 'Recv-Q', 'Send-Q', |
57 | 'Local Address', 'Foreign Address', 'State'] |
58 | - tbl_v6 = PrettyTable(fields_v6) |
59 | + tbl_v6 = SimpleTable(fields_v6) |
60 | for (n, r) in enumerate(routes.get('ipv6')): |
61 | route_id = str(n) |
62 | tbl_v6.add_row([route_id, r['proto'], |
63 | diff --git a/cloudinit/simpletable.py b/cloudinit/simpletable.py |
64 | new file mode 100644 |
65 | index 0000000..9060322 |
66 | --- /dev/null |
67 | +++ b/cloudinit/simpletable.py |
68 | @@ -0,0 +1,62 @@ |
69 | +# Copyright (C) 2017 Amazon.com, Inc. or its affiliates |
70 | +# |
71 | +# Author: Ethan Faust <efaust@amazon.com> |
72 | +# Author: Andrew Jorgensen <ajorgens@amazon.com> |
73 | +# |
74 | +# This file is part of cloud-init. See LICENSE file for license information. |
75 | + |
76 | + |
77 | +class SimpleTable(object): |
78 | + """A minimal implementation of PrettyTable |
79 | + for distribution with cloud-init. |
80 | + """ |
81 | + |
82 | + def __init__(self, fields): |
83 | + self.fields = fields |
84 | + self.rows = [] |
85 | + |
86 | + # initialize list of 0s the same length |
87 | + # as the number of fields |
88 | + self.column_widths = [0] * len(self.fields) |
89 | + self.update_column_widths(fields) |
90 | + |
91 | + def update_column_widths(self, values): |
92 | + for i, value in enumerate(values): |
93 | + self.column_widths[i] = max( |
94 | + len(value), |
95 | + self.column_widths[i]) |
96 | + |
97 | + def add_row(self, values): |
98 | + if len(values) > len(self.fields): |
99 | + raise TypeError('too many values') |
100 | + values = [str(value) for value in values] |
101 | + self.rows.append(values) |
102 | + self.update_column_widths(values) |
103 | + |
104 | + def _hdiv(self): |
105 | + """Returns a horizontal divider for the table.""" |
106 | + return '+' + '+'.join( |
107 | + ['-' * (w + 2) for w in self.column_widths]) + '+' |
108 | + |
109 | + def _row(self, row): |
110 | + """Returns a formatted row.""" |
111 | + return '|' + '|'.join( |
112 | + [col.center(self.column_widths[i] + 2) |
113 | + for i, col in enumerate(row)]) + '|' |
114 | + |
115 | + def __str__(self): |
116 | + """Returns a string representation of the table with lines around. |
117 | + |
118 | + +-----+-----+ |
119 | + | one | two | |
120 | + +-----+-----+ |
121 | + | 1 | 2 | |
122 | + | 01 | 10 | |
123 | + +-----+-----+ |
124 | + """ |
125 | + lines = [self._hdiv(), self._row(self.fields), self._hdiv()] |
126 | + lines += [self._row(r) for r in self.rows] + [self._hdiv()] |
127 | + return '\n'.join(lines) |
128 | + |
129 | + def get_string(self): |
130 | + return repr(self) |
131 | diff --git a/cloudinit/tests/test_simpletable.py b/cloudinit/tests/test_simpletable.py |
132 | new file mode 100644 |
133 | index 0000000..96bc24c |
134 | --- /dev/null |
135 | +++ b/cloudinit/tests/test_simpletable.py |
136 | @@ -0,0 +1,100 @@ |
137 | +# Copyright (C) 2017 Amazon.com, Inc. or its affiliates |
138 | +# |
139 | +# Author: Andrew Jorgensen <ajorgens@amazon.com> |
140 | +# |
141 | +# This file is part of cloud-init. See LICENSE file for license information. |
142 | +"""Tests that SimpleTable works just like PrettyTable for cloud-init. |
143 | + |
144 | +Not all possible PrettyTable cases are tested because we're not trying to |
145 | +reimplement the entire library, only the minimal parts we actually use. |
146 | +""" |
147 | + |
148 | +from cloudinit.simpletable import SimpleTable |
149 | +from cloudinit.tests.helpers import CiTestCase |
150 | + |
151 | +# Examples rendered by cloud-init using PrettyTable |
152 | +NET_DEVICE_FIELDS = ( |
153 | + 'Device', 'Up', 'Address', 'Mask', 'Scope', 'Hw-Address') |
154 | +NET_DEVICE_ROWS = ( |
155 | + ('ens3', True, '172.31.4.203', '255.255.240.0', '.', '0a:1f:07:15:98:70'), |
156 | + ('ens3', True, 'fe80::81f:7ff:fe15:9870/64', '.', 'link', |
157 | + '0a:1f:07:15:98:70'), |
158 | + ('lo', True, '127.0.0.1', '255.0.0.0', '.', '.'), |
159 | + ('lo', True, '::1/128', '.', 'host', '.'), |
160 | +) |
161 | +NET_DEVICE_TABLE = """\ |
162 | ++--------+------+----------------------------+---------------+-------+-------------------+ |
163 | +| Device | Up | Address | Mask | Scope | Hw-Address | |
164 | ++--------+------+----------------------------+---------------+-------+-------------------+ |
165 | +| ens3 | True | 172.31.4.203 | 255.255.240.0 | . | 0a:1f:07:15:98:70 | |
166 | +| ens3 | True | fe80::81f:7ff:fe15:9870/64 | . | link | 0a:1f:07:15:98:70 | |
167 | +| lo | True | 127.0.0.1 | 255.0.0.0 | . | . | |
168 | +| lo | True | ::1/128 | . | host | . | |
169 | ++--------+------+----------------------------+---------------+-------+-------------------+""" # noqa: E501 |
170 | +ROUTE_IPV4_FIELDS = ( |
171 | + 'Route', 'Destination', 'Gateway', 'Genmask', 'Interface', 'Flags') |
172 | +ROUTE_IPV4_ROWS = ( |
173 | + ('0', '0.0.0.0', '172.31.0.1', '0.0.0.0', 'ens3', 'UG'), |
174 | + ('1', '169.254.0.0', '0.0.0.0', '255.255.0.0', 'ens3', 'U'), |
175 | + ('2', '172.31.0.0', '0.0.0.0', '255.255.240.0', 'ens3', 'U'), |
176 | +) |
177 | +ROUTE_IPV4_TABLE = """\ |
178 | ++-------+-------------+------------+---------------+-----------+-------+ |
179 | +| Route | Destination | Gateway | Genmask | Interface | Flags | |
180 | ++-------+-------------+------------+---------------+-----------+-------+ |
181 | +| 0 | 0.0.0.0 | 172.31.0.1 | 0.0.0.0 | ens3 | UG | |
182 | +| 1 | 169.254.0.0 | 0.0.0.0 | 255.255.0.0 | ens3 | U | |
183 | +| 2 | 172.31.0.0 | 0.0.0.0 | 255.255.240.0 | ens3 | U | |
184 | ++-------+-------------+------------+---------------+-----------+-------+""" |
185 | + |
186 | +AUTHORIZED_KEYS_FIELDS = ( |
187 | + 'Keytype', 'Fingerprint (md5)', 'Options', 'Comment') |
188 | +AUTHORIZED_KEYS_ROWS = ( |
189 | + ('ssh-rsa', '24:c7:41:49:47:12:31:a0:de:6f:62:79:9b:13:06:36', '-', |
190 | + 'ajorgens'), |
191 | +) |
192 | +AUTHORIZED_KEYS_TABLE = """\ |
193 | ++---------+-------------------------------------------------+---------+----------+ |
194 | +| Keytype | Fingerprint (md5) | Options | Comment | |
195 | ++---------+-------------------------------------------------+---------+----------+ |
196 | +| ssh-rsa | 24:c7:41:49:47:12:31:a0:de:6f:62:79:9b:13:06:36 | - | ajorgens | |
197 | ++---------+-------------------------------------------------+---------+----------+""" # noqa: E501 |
198 | + |
199 | +# from prettytable import PrettyTable |
200 | +# pt = PrettyTable(('HEADER',)) |
201 | +# print(pt) |
202 | +NO_ROWS_FIELDS = ('HEADER',) |
203 | +NO_ROWS_TABLE = """\ |
204 | ++--------+ |
205 | +| HEADER | |
206 | ++--------+ |
207 | ++--------+""" |
208 | + |
209 | + |
210 | +class TestSimpleTable(CiTestCase): |
211 | + |
212 | + def test_no_rows(self): |
213 | + """An empty table is rendered as PrettyTable would have done it.""" |
214 | + table = SimpleTable(NO_ROWS_FIELDS) |
215 | + self.assertEqual(str(table), NO_ROWS_TABLE) |
216 | + |
217 | + def test_net_dev(self): |
218 | + """Net device info is rendered as it was with PrettyTable.""" |
219 | + table = SimpleTable(NET_DEVICE_FIELDS) |
220 | + for row in NET_DEVICE_ROWS: |
221 | + table.add_row(row) |
222 | + self.assertEqual(str(table), NET_DEVICE_TABLE) |
223 | + |
224 | + def test_route_ipv4(self): |
225 | + """Route IPv4 info is rendered as it was with PrettyTable.""" |
226 | + table = SimpleTable(ROUTE_IPV4_FIELDS) |
227 | + for row in ROUTE_IPV4_ROWS: |
228 | + table.add_row(row) |
229 | + self.assertEqual(str(table), ROUTE_IPV4_TABLE) |
230 | + |
231 | + def test_authorized_keys(self): |
232 | + """SSH authorized keys are rendered as they were with PrettyTable.""" |
233 | + table = SimpleTable(AUTHORIZED_KEYS_FIELDS) |
234 | + for row in AUTHORIZED_KEYS_ROWS: |
235 | + table.add_row(row) |
236 | + self.assertEqual(str(table), AUTHORIZED_KEYS_TABLE) |
237 | diff --git a/packages/pkg-deps.json b/packages/pkg-deps.json |
238 | index 822d29d..72409dd 100644 |
239 | --- a/packages/pkg-deps.json |
240 | +++ b/packages/pkg-deps.json |
241 | @@ -34,9 +34,6 @@ |
242 | "jsonschema" : { |
243 | "3" : "python34-jsonschema" |
244 | }, |
245 | - "prettytable" : { |
246 | - "3" : "python34-prettytable" |
247 | - }, |
248 | "pyflakes" : { |
249 | "2" : "pyflakes", |
250 | "3" : "python34-pyflakes" |
251 | diff --git a/requirements.txt b/requirements.txt |
252 | index 61d1e90..dd10d85 100644 |
253 | --- a/requirements.txt |
254 | +++ b/requirements.txt |
255 | @@ -3,9 +3,6 @@ |
256 | # Used for untemplating any files or strings with parameters. |
257 | jinja2 |
258 | |
259 | -# This is used for any pretty printing of tabular data. |
260 | -PrettyTable |
261 | - |
262 | # This one is currently only used by the MAAS datasource. If that |
263 | # datasource is removed, this is no longer needed |
264 | oauthlib |
265 | diff --git a/tools/build-on-freebsd b/tools/build-on-freebsd |
266 | index ff9153a..d23fde2 100755 |
267 | --- a/tools/build-on-freebsd |
268 | +++ b/tools/build-on-freebsd |
269 | @@ -18,7 +18,6 @@ pkgs=" |
270 | py27-jsonpatch |
271 | py27-jsonpointer |
272 | py27-oauthlib |
273 | - py27-prettytable |
274 | py27-requests |
275 | py27-serial |
276 | py27-six |
277 | diff --git a/tox.ini b/tox.ini |
278 | index 776f425..aef1f84 100644 |
279 | --- a/tox.ini |
280 | +++ b/tox.ini |
281 | @@ -64,7 +64,6 @@ deps = |
282 | # requirements |
283 | jinja2==2.8 |
284 | pyyaml==3.11 |
285 | - PrettyTable==0.7.2 |
286 | oauthlib==1.0.3 |
287 | pyserial==3.0.1 |
288 | configobj==5.0.6 |
289 | @@ -89,7 +88,6 @@ deps = |
290 | argparse==1.2.1 |
291 | jinja2==2.2.1 |
292 | pyyaml==3.10 |
293 | - PrettyTable==0.7.2 |
294 | oauthlib==0.6.0 |
295 | configobj==4.6.0 |
296 | requests==2.6.0 |
297 | @@ -105,7 +103,6 @@ deps = |
298 | argparse==1.3.0 |
299 | jinja2==2.8 |
300 | PyYAML==3.11 |
301 | - PrettyTable==0.7.2 |
302 | oauthlib==0.7.2 |
303 | configobj==5.0.6 |
304 | requests==2.11.1 |
Noticed a mention of PrettyTable in the podcast Scott mentioned on the mailing list, so I thought I'd better post what Amazon Linux uses in place of PrettyTable.
No unit tests written yet, but hopefully I'll be able to take some time to do that soon.
This is an Amazon contribution, not an individual contribution, but I got permission from the original author anyway.