Merge ~ahasenack/ubuntu/+source/python-certbot:bionic-certbot-backport-1837673 into ubuntu/+source/python-certbot:ubuntu/bionic-devel

Proposed by Andreas Hasenack
Status: Superseded
Proposed branch: ~ahasenack/ubuntu/+source/python-certbot:bionic-certbot-backport-1837673
Merge into: ubuntu/+source/python-certbot:ubuntu/bionic-devel
Diff against target: 6455 lines (+2780/-562)
74 files modified
PKG-INFO (+6/-4)
README.rst (+2/-2)
certbot.egg-info/PKG-INFO (+6/-4)
certbot.egg-info/SOURCES.txt (+9/-0)
certbot.egg-info/requires.txt (+6/-2)
certbot/__init__.py (+1/-1)
certbot/account.py (+121/-13)
certbot/auth_handler.py (+14/-12)
certbot/cert_manager.py (+23/-16)
certbot/cli.py (+97/-29)
certbot/client.py (+92/-30)
certbot/configuration.py (+5/-1)
certbot/constants.py (+17/-2)
certbot/crypto_util.py (+87/-63)
certbot/display/ops.py (+26/-6)
certbot/display/util.py (+9/-10)
certbot/eff.py (+8/-5)
certbot/error_handler.py (+10/-6)
certbot/errors.py (+4/-0)
certbot/hooks.py (+10/-7)
certbot/interfaces.py (+81/-3)
certbot/log.py (+1/-2)
certbot/main.py (+129/-31)
certbot/plugins/common.py (+7/-3)
certbot/plugins/disco.py (+7/-1)
certbot/plugins/disco_test.py (+2/-1)
certbot/plugins/enhancements.py (+164/-0)
certbot/plugins/enhancements_test.py (+65/-0)
certbot/plugins/manual.py (+4/-1)
certbot/plugins/selection.py (+52/-8)
certbot/plugins/selection_test.py (+49/-2)
certbot/plugins/standalone.py (+21/-8)
certbot/plugins/standalone_test.py (+13/-6)
certbot/plugins/storage.py (+119/-0)
certbot/plugins/storage_test.py (+117/-0)
certbot/plugins/util.py (+2/-2)
certbot/plugins/util_test.py (+3/-6)
certbot/plugins/webroot.py (+10/-7)
certbot/renewal.py (+26/-14)
certbot/reverter.py (+8/-6)
certbot/storage.py (+13/-9)
certbot/tests/account_test.py (+129/-1)
certbot/tests/auth_handler_test.py (+3/-1)
certbot/tests/cert_manager_test.py (+101/-2)
certbot/tests/cli_test.py (+2/-1)
certbot/tests/client_test.py (+77/-5)
certbot/tests/crypto_util_test.py (+10/-0)
certbot/tests/display/completer_test.py (+2/-1)
certbot/tests/display/ops_test.py (+48/-2)
certbot/tests/eff_test.py (+18/-0)
certbot/tests/error_handler_test.py (+4/-2)
certbot/tests/hook_test.py (+11/-10)
certbot/tests/log_test.py (+7/-6)
certbot/tests/main_test.py (+259/-26)
certbot/tests/renewupdater_test.py (+125/-0)
certbot/tests/reporter_test.py (+9/-9)
certbot/tests/storage_test.py (+37/-11)
certbot/tests/testdata/cert-nosans_nistp256.pem (+11/-0)
certbot/tests/testdata/csr-nosans_nistp256.pem (+8/-0)
certbot/tests/testdata/nistp256_key.pem (+5/-0)
certbot/updater.py (+122/-0)
certbot/util.py (+10/-6)
debian/changelog (+51/-0)
debian/control (+8/-6)
debian/patches/0001-remove-external-images.patch (+2/-2)
debian/python3-certbot.lintian-overrides (+3/-0)
dev/null (+0/-41)
docs/cli-help.txt (+90/-17)
docs/conf.py (+1/-0)
docs/contributing.rst (+95/-58)
docs/install.rst (+41/-14)
docs/packaging.rst (+16/-2)
docs/using.rst (+17/-7)
setup.py (+12/-9)
Reviewer Review Type Date Requested Status
Canonical Server Pending
Review via email: mp+374373@code.launchpad.net

This proposal has been superseded by a proposal from 2019-10-18.

Description of the change

Bileto ticket, with ppa (still building): https://bileto.ubuntu.com/#/ticket/3827

Backport cosmic's certbot package to bionic. The only change I added was removing the letsencrypt.postrm script, something that debian did in later releases and seemed sane, as that is a transitional package (both in current bionic, and in this proposed update).
The cosmic package has a lintian override that seems to be a no-op in bionic. lintian on bionic at least doesn't seem to have that error/warning. I opted to include it anyway, to reduce the delta with the origin package (the cosmic one).

To post a comment you must log in.

Unmerged commits

97de715... by Andreas Hasenack

update-maintainer

5896c00... by Andreas Hasenack

changelog

7069d90... by Andreas Hasenack

  * Backport to bionic (LP: #1837673):
    - d/letsencrypt.postrm: purging the transitional package shouldn't
      remove the logs (Closes: #921423)

2248f0b... by Harlan Lieberman-Berg

Import patches-unapplied version 0.27.0-1 to debian/sid

Imported using git-ubuntu import.

Changelog parent: a2ec5fc726299dac7a259ad8b7d9b5ac3a9946b9

New changelog entries:
  * New upstream version 0.27.0
  * Refresh patch after upstream migration to codecov
  * Bump python-sphinx requirement defensively; bump S-V with no changes
  * Bump dep on python-acme to 0.26.0~

a2ec5fc... by Harlan Lieberman-Berg

Import patches-unapplied version 0.26.1-1 to debian/sid

Imported using git-ubuntu import.

Changelog parent: 06bbf01f039c6f3f47b8919f7d7087fd733645ce

New changelog entries:
  * New upstream release.

06bbf01... by Harlan Lieberman-Berg

Import patches-unapplied version 0.26.0-1 to debian/sid

Imported using git-ubuntu import.

Changelog parent: d5a6a55016ec7327c710c06bb8b7491a27c5a928

New changelog entries:
  * New upstream version 0.26.0
  * Bump S-V; add R-R-R: no

d5a6a55... by Harlan Lieberman-Berg

Import patches-unapplied version 0.25.0-1 to debian/sid

Imported using git-ubuntu import.

Changelog parent: d57b9c7ebc60b04a67e78a4a96a201c5c5edc4f8

New changelog entries:
  * New upstream version 0.25.0
  * Bump python-acme dep version.

d57b9c7... by Harlan Lieberman-Berg

Import patches-unapplied version 0.24.0-2 to debian/sid

Imported using git-ubuntu import.

Changelog parent: b1d8dcd28ffc56f03b2a29b7516e2511f9ee9596

New changelog entries:
  * Update team email address. (Closes: #899858)

b1d8dcd... by Harlan Lieberman-Berg

Import patches-unapplied version 0.24.0-1 to debian/sid

Imported using git-ubuntu import.

Changelog parent: 0d21435e8ce4e37943d832ee032bc21a84ba1981

New changelog entries:
  * Add OR to dep on python-distutils for stretch-bpo
  * New upstream version 0.24.0
  * Bump version dep on python3-acme

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/PKG-INFO b/PKG-INFO
2index 68ad466..c91ad37 100644
3--- a/PKG-INFO
4+++ b/PKG-INFO
5@@ -1,6 +1,6 @@
6 Metadata-Version: 2.1
7 Name: certbot
8-Version: 0.23.0
9+Version: 0.27.0
10 Summary: ACME client
11 Home-page: https://github.com/letsencrypt/letsencrypt
12 Author: Certbot Project
13@@ -115,8 +115,8 @@ Description: .. This file contains a series of comments that are used to include
14 :target: https://travis-ci.org/certbot/certbot
15 :alt: Travis CI status
16
17- .. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
18- :target: https://coveralls.io/r/certbot/certbot
19+ .. |coverage| image:: https://codecov.io/gh/certbot/certbot/branch/master/graph/badge.svg
20+ :target: https://codecov.io/gh/certbot/certbot
21 :alt: Coverage status
22
23 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
24@@ -169,7 +169,7 @@ Description: .. This file contains a series of comments that are used to include
25 For extensive documentation on using and contributing to Certbot, go to https://certbot.eff.org/docs. If you would like to contribute to the project or run the latest code from git, you should read our `developer guide <https://certbot.eff.org/docs/contributing.html>`_.
26
27 Platform: UNKNOWN
28-Classifier: Development Status :: 3 - Alpha
29+Classifier: Development Status :: 5 - Production/Stable
30 Classifier: Environment :: Console
31 Classifier: Environment :: Console :: Curses
32 Classifier: Intended Audience :: System Administrators
33@@ -182,6 +182,7 @@ Classifier: Programming Language :: Python :: 3
34 Classifier: Programming Language :: Python :: 3.4
35 Classifier: Programming Language :: Python :: 3.5
36 Classifier: Programming Language :: Python :: 3.6
37+Classifier: Programming Language :: Python :: 3.7
38 Classifier: Topic :: Internet :: WWW/HTTP
39 Classifier: Topic :: Security
40 Classifier: Topic :: System :: Installation/Setup
41@@ -189,5 +190,6 @@ Classifier: Topic :: System :: Networking
42 Classifier: Topic :: System :: Systems Administration
43 Classifier: Topic :: Utilities
44 Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
45+Provides-Extra: dev3
46 Provides-Extra: docs
47 Provides-Extra: dev
48diff --git a/README.rst b/README.rst
49index a18028a..0dbe1cd 100644
50--- a/README.rst
51+++ b/README.rst
52@@ -107,8 +107,8 @@ ACME working area in github: https://github.com/ietf-wg-acme/acme
53 :target: https://travis-ci.org/certbot/certbot
54 :alt: Travis CI status
55
56-.. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
57- :target: https://coveralls.io/r/certbot/certbot
58+.. |coverage| image:: https://codecov.io/gh/certbot/certbot/branch/master/graph/badge.svg
59+ :target: https://codecov.io/gh/certbot/certbot
60 :alt: Coverage status
61
62 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
63diff --git a/certbot.egg-info/PKG-INFO b/certbot.egg-info/PKG-INFO
64index 68ad466..c91ad37 100644
65--- a/certbot.egg-info/PKG-INFO
66+++ b/certbot.egg-info/PKG-INFO
67@@ -1,6 +1,6 @@
68 Metadata-Version: 2.1
69 Name: certbot
70-Version: 0.23.0
71+Version: 0.27.0
72 Summary: ACME client
73 Home-page: https://github.com/letsencrypt/letsencrypt
74 Author: Certbot Project
75@@ -115,8 +115,8 @@ Description: .. This file contains a series of comments that are used to include
76 :target: https://travis-ci.org/certbot/certbot
77 :alt: Travis CI status
78
79- .. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
80- :target: https://coveralls.io/r/certbot/certbot
81+ .. |coverage| image:: https://codecov.io/gh/certbot/certbot/branch/master/graph/badge.svg
82+ :target: https://codecov.io/gh/certbot/certbot
83 :alt: Coverage status
84
85 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
86@@ -169,7 +169,7 @@ Description: .. This file contains a series of comments that are used to include
87 For extensive documentation on using and contributing to Certbot, go to https://certbot.eff.org/docs. If you would like to contribute to the project or run the latest code from git, you should read our `developer guide <https://certbot.eff.org/docs/contributing.html>`_.
88
89 Platform: UNKNOWN
90-Classifier: Development Status :: 3 - Alpha
91+Classifier: Development Status :: 5 - Production/Stable
92 Classifier: Environment :: Console
93 Classifier: Environment :: Console :: Curses
94 Classifier: Intended Audience :: System Administrators
95@@ -182,6 +182,7 @@ Classifier: Programming Language :: Python :: 3
96 Classifier: Programming Language :: Python :: 3.4
97 Classifier: Programming Language :: Python :: 3.5
98 Classifier: Programming Language :: Python :: 3.6
99+Classifier: Programming Language :: Python :: 3.7
100 Classifier: Topic :: Internet :: WWW/HTTP
101 Classifier: Topic :: Security
102 Classifier: Topic :: System :: Installation/Setup
103@@ -189,5 +190,6 @@ Classifier: Topic :: System :: Networking
104 Classifier: Topic :: System :: Systems Administration
105 Classifier: Topic :: Utilities
106 Requires-Python: >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
107+Provides-Extra: dev3
108 Provides-Extra: docs
109 Provides-Extra: dev
110diff --git a/certbot.egg-info/SOURCES.txt b/certbot.egg-info/SOURCES.txt
111index 9283644..b41c65d 100644
112--- a/certbot.egg-info/SOURCES.txt
113+++ b/certbot.egg-info/SOURCES.txt
114@@ -31,6 +31,7 @@ certbot/reporter.py
115 certbot/reverter.py
116 certbot/ssl-dhparams.pem
117 certbot/storage.py
118+certbot/updater.py
119 certbot/util.py
120 certbot.egg-info/PKG-INFO
121 certbot.egg-info/SOURCES.txt
122@@ -55,6 +56,8 @@ certbot/plugins/dns_common_lexicon_test.py
123 certbot/plugins/dns_common_test.py
124 certbot/plugins/dns_test_common.py
125 certbot/plugins/dns_test_common_lexicon.py
126+certbot/plugins/enhancements.py
127+certbot/plugins/enhancements_test.py
128 certbot/plugins/manual.py
129 certbot/plugins/manual_test.py
130 certbot/plugins/null.py
131@@ -63,6 +66,8 @@ certbot/plugins/selection.py
132 certbot/plugins/selection_test.py
133 certbot/plugins/standalone.py
134 certbot/plugins/standalone_test.py
135+certbot/plugins/storage.py
136+certbot/plugins/storage_test.py
137 certbot/plugins/util.py
138 certbot/plugins/util_test.py
139 certbot/plugins/webroot.py
140@@ -86,6 +91,7 @@ certbot/tests/main_test.py
141 certbot/tests/notify_test.py
142 certbot/tests/ocsp_test.py
143 certbot/tests/renewal_test.py
144+certbot/tests/renewupdater_test.py
145 certbot/tests/reporter_test.py
146 certbot/tests/reverter_test.py
147 certbot/tests/storage_test.py
148@@ -98,6 +104,7 @@ certbot/tests/display/ops_test.py
149 certbot/tests/display/util_test.py
150 certbot/tests/testdata/README
151 certbot/tests/testdata/cert-5sans_512.pem
152+certbot/tests/testdata/cert-nosans_nistp256.pem
153 certbot/tests/testdata/cert-san_512.pem
154 certbot/tests/testdata/cert_2048.pem
155 certbot/tests/testdata/cert_512.pem
156@@ -109,9 +116,11 @@ certbot/tests/testdata/csr-6sans_512.pem
157 certbot/tests/testdata/csr-nonames_512.pem
158 certbot/tests/testdata/csr-nosans_512.conf
159 certbot/tests/testdata/csr-nosans_512.pem
160+certbot/tests/testdata/csr-nosans_nistp256.pem
161 certbot/tests/testdata/csr-san_512.pem
162 certbot/tests/testdata/csr_512.der
163 certbot/tests/testdata/csr_512.pem
164+certbot/tests/testdata/nistp256_key.pem
165 certbot/tests/testdata/os-release
166 certbot/tests/testdata/rsa2048_key.pem
167 certbot/tests/testdata/rsa256_key.pem
168diff --git a/certbot.egg-info/requires.txt b/certbot.egg-info/requires.txt
169index a3c0ea0..33ce4fe 100644
170--- a/certbot.egg-info/requires.txt
171+++ b/certbot.egg-info/requires.txt
172@@ -1,4 +1,4 @@
173-acme>0.21.1
174+acme>=0.26.0
175 ConfigArgParse>=0.9.3
176 configobj
177 cryptography>=1.2
178@@ -23,7 +23,11 @@ tox
179 twine
180 wheel
181
182+[dev3]
183+mypy
184+typing
185+
186 [docs]
187 repoze.sphinx.autointerface
188-Sphinx<=1.5.6,>=1.0
189+Sphinx>=1.6
190 sphinx_rtd_theme
191diff --git a/certbot/__init__.py b/certbot/__init__.py
192index 53ba17e..d6fee2e 100644
193--- a/certbot/__init__.py
194+++ b/certbot/__init__.py
195@@ -1,4 +1,4 @@
196 """Certbot client."""
197
198 # version number like 1.2.3a0, must have at least 2 parts, like 1.2
199-__version__ = '0.23.0'
200+__version__ = '0.27.0'
201diff --git a/certbot/account.py b/certbot/account.py
202index 70d9a7f..59ceb42 100644
203--- a/certbot/account.py
204+++ b/certbot/account.py
205@@ -1,5 +1,6 @@
206 """Creates ACME accounts for server."""
207 import datetime
208+import functools
209 import hashlib
210 import logging
211 import os
212@@ -16,6 +17,7 @@ import zope.component
213 from acme import fields as acme_fields
214 from acme import messages
215
216+from certbot import constants
217 from certbot import errors
218 from certbot import interfaces
219 from certbot import util
220@@ -142,7 +144,11 @@ class AccountFileStorage(interfaces.AccountStorage):
221 self.config.strict_permissions)
222
223 def _account_dir_path(self, account_id):
224- return os.path.join(self.config.accounts_dir, account_id)
225+ return self._account_dir_path_for_server_path(account_id, self.config.server_path)
226+
227+ def _account_dir_path_for_server_path(self, account_id, server_path):
228+ accounts_dir = self.config.accounts_dir_for_server_path(server_path)
229+ return os.path.join(accounts_dir, account_id)
230
231 @classmethod
232 def _regr_path(cls, account_dir_path):
233@@ -156,25 +162,67 @@ class AccountFileStorage(interfaces.AccountStorage):
234 def _metadata_path(cls, account_dir_path):
235 return os.path.join(account_dir_path, "meta.json")
236
237- def find_all(self):
238+ def _find_all_for_server_path(self, server_path):
239+ accounts_dir = self.config.accounts_dir_for_server_path(server_path)
240 try:
241- candidates = os.listdir(self.config.accounts_dir)
242+ candidates = os.listdir(accounts_dir)
243 except OSError:
244 return []
245
246 accounts = []
247 for account_id in candidates:
248 try:
249- accounts.append(self.load(account_id))
250+ accounts.append(self._load_for_server_path(account_id, server_path))
251 except errors.AccountStorageError:
252 logger.debug("Account loading problem", exc_info=True)
253+
254+ if not accounts and server_path in constants.LE_REUSE_SERVERS:
255+ # find all for the next link down
256+ prev_server_path = constants.LE_REUSE_SERVERS[server_path]
257+ prev_accounts = self._find_all_for_server_path(prev_server_path)
258+ # if we found something, link to that
259+ if prev_accounts:
260+ try:
261+ self._symlink_to_accounts_dir(prev_server_path, server_path)
262+ except OSError:
263+ return []
264+ accounts = prev_accounts
265 return accounts
266
267- def load(self, account_id):
268- account_dir_path = self._account_dir_path(account_id)
269- if not os.path.isdir(account_dir_path):
270- raise errors.AccountNotFound(
271- "Account at %s does not exist" % account_dir_path)
272+ def find_all(self):
273+ return self._find_all_for_server_path(self.config.server_path)
274+
275+ def _symlink_to_account_dir(self, prev_server_path, server_path, account_id):
276+ prev_account_dir = self._account_dir_path_for_server_path(account_id, prev_server_path)
277+ new_account_dir = self._account_dir_path_for_server_path(account_id, server_path)
278+ os.symlink(prev_account_dir, new_account_dir)
279+
280+ def _symlink_to_accounts_dir(self, prev_server_path, server_path):
281+ accounts_dir = self.config.accounts_dir_for_server_path(server_path)
282+ if os.path.islink(accounts_dir):
283+ os.unlink(accounts_dir)
284+ else:
285+ os.rmdir(accounts_dir)
286+ prev_account_dir = self.config.accounts_dir_for_server_path(prev_server_path)
287+ os.symlink(prev_account_dir, accounts_dir)
288+
289+ def _load_for_server_path(self, account_id, server_path):
290+ account_dir_path = self._account_dir_path_for_server_path(account_id, server_path)
291+ if not os.path.isdir(account_dir_path): # isdir is also true for symlinks
292+ if server_path in constants.LE_REUSE_SERVERS:
293+ prev_server_path = constants.LE_REUSE_SERVERS[server_path]
294+ prev_loaded_account = self._load_for_server_path(account_id, prev_server_path)
295+ # we didn't error so we found something, so create a symlink to that
296+ accounts_dir = self.config.accounts_dir_for_server_path(server_path)
297+ # If accounts_dir isn't empty, make an account specific symlink
298+ if os.listdir(accounts_dir):
299+ self._symlink_to_account_dir(prev_server_path, server_path, account_id)
300+ else:
301+ self._symlink_to_accounts_dir(prev_server_path, server_path)
302+ return prev_loaded_account
303+ else:
304+ raise errors.AccountNotFound(
305+ "Account at %s does not exist" % account_dir_path)
306
307 try:
308 with open(self._regr_path(account_dir_path)) as regr_file:
309@@ -193,6 +241,9 @@ class AccountFileStorage(interfaces.AccountStorage):
310 account_id, acc.id))
311 return acc
312
313+ def load(self, account_id):
314+ return self._load_for_server_path(account_id, self.config.server_path)
315+
316 def save(self, account, acme):
317 self._save(account, acme, regr_only=False)
318
319@@ -214,7 +265,61 @@ class AccountFileStorage(interfaces.AccountStorage):
320 if not os.path.isdir(account_dir_path):
321 raise errors.AccountNotFound(
322 "Account at %s does not exist" % account_dir_path)
323- shutil.rmtree(account_dir_path)
324+ # Step 1: Delete account specific links and the directory
325+ self._delete_account_dir_for_server_path(account_id, self.config.server_path)
326+
327+ # Step 2: Remove any accounts links and directories that are now empty
328+ if not os.listdir(self.config.accounts_dir):
329+ self._delete_accounts_dir_for_server_path(self.config.server_path)
330+
331+ def _delete_account_dir_for_server_path(self, account_id, server_path):
332+ link_func = functools.partial(self._account_dir_path_for_server_path, account_id)
333+ nonsymlinked_dir = self._delete_links_and_find_target_dir(server_path, link_func)
334+ shutil.rmtree(nonsymlinked_dir)
335+
336+ def _delete_accounts_dir_for_server_path(self, server_path):
337+ link_func = self.config.accounts_dir_for_server_path
338+ nonsymlinked_dir = self._delete_links_and_find_target_dir(server_path, link_func)
339+ os.rmdir(nonsymlinked_dir)
340+
341+ def _delete_links_and_find_target_dir(self, server_path, link_func):
342+ """Delete symlinks and return the nonsymlinked directory path.
343+
344+ :param str server_path: file path based on server
345+ :param callable link_func: callable that returns possible links
346+ given a server_path
347+
348+ :returns: the final, non-symlinked target
349+ :rtype: str
350+
351+ """
352+ dir_path = link_func(server_path)
353+
354+ # does an appropriate directory link to me? if so, make sure that's gone
355+ reused_servers = {}
356+ for k in constants.LE_REUSE_SERVERS:
357+ reused_servers[constants.LE_REUSE_SERVERS[k]] = k
358+
359+ # is there a next one up?
360+ possible_next_link = True
361+ while possible_next_link:
362+ possible_next_link = False
363+ if server_path in reused_servers:
364+ next_server_path = reused_servers[server_path]
365+ next_dir_path = link_func(next_server_path)
366+ if os.path.islink(next_dir_path) and os.readlink(next_dir_path) == dir_path:
367+ possible_next_link = True
368+ server_path = next_server_path
369+ dir_path = next_dir_path
370+
371+ # if there's not a next one up to delete, then delete me
372+ # and whatever I link to
373+ while os.path.islink(dir_path):
374+ target = os.readlink(dir_path)
375+ os.unlink(dir_path)
376+ dir_path = target
377+
378+ return dir_path
379
380 def _save(self, account, acme, regr_only):
381 account_dir_path = self._account_dir_path(account.id)
382@@ -230,9 +335,12 @@ class AccountFileStorage(interfaces.AccountStorage):
383 if hasattr(acme.directory, "new-authz"):
384 regr = RegistrationResourceWithNewAuthzrURI(
385 new_authzr_uri=acme.directory.new_authz,
386- body=regr.body,
387- uri=regr.uri,
388- terms_of_service=regr.terms_of_service)
389+ body={},
390+ uri=regr.uri)
391+ else:
392+ regr = messages.RegistrationResource(
393+ body={},
394+ uri=regr.uri)
395 regr_file.write(regr.json_dumps())
396 if not regr_only:
397 with util.safe_open(self._key_path(account_dir_path),
398diff --git a/certbot/auth_handler.py b/certbot/auth_handler.py
399index 9d7c75f..e7d658b 100644
400--- a/certbot/auth_handler.py
401+++ b/certbot/auth_handler.py
402@@ -8,7 +8,9 @@ import zope.component
403
404 from acme import challenges
405 from acme import messages
406-
407+# pylint: disable=unused-import, no-name-in-module
408+from acme.magic_typing import DefaultDict, Dict, List, Set, Collection
409+# pylint: enable=unused-import, no-name-in-module
410 from certbot import achallenges
411 from certbot import errors
412 from certbot import error_handler
413@@ -117,7 +119,7 @@ class AuthHandler(object):
414
415 def _solve_challenges(self, aauthzrs):
416 """Get Responses for challenges from authenticators."""
417- resp = []
418+ resp = [] # type: Collection[acme.challenges.ChallengeResponse]
419 all_achalls = self._get_all_achalls(aauthzrs)
420 try:
421 if all_achalls:
422@@ -133,10 +135,9 @@ class AuthHandler(object):
423
424 def _get_all_achalls(self, aauthzrs):
425 """Return all active challenges."""
426- all_achalls = []
427+ all_achalls = [] # type: Collection[challenges.ChallengeResponse]
428 for aauthzr in aauthzrs:
429 all_achalls.extend(aauthzr.achalls)
430-
431 return all_achalls
432
433 def _respond(self, aauthzrs, resp, best_effort):
434@@ -146,7 +147,8 @@ class AuthHandler(object):
435
436 """
437 # TODO: chall_update is a dirty hack to get around acme-spec #105
438- chall_update = dict()
439+ chall_update = dict() \
440+ # type: Dict[int, List[achallenges.KeyAuthorizationAnnotatedChallenge]]
441 self._send_responses(aauthzrs, resp, chall_update)
442
443 # Check for updated status...
444@@ -189,7 +191,7 @@ class AuthHandler(object):
445 return active_achalls
446
447 def _poll_challenges(self, aauthzrs, chall_update,
448- best_effort, min_sleep=3, max_rounds=15):
449+ best_effort, min_sleep=3, max_rounds=30):
450 """Wait for all challenge results to be determined."""
451 indices_to_check = set(chall_update.keys())
452 comp_indices = set()
453@@ -198,7 +200,7 @@ class AuthHandler(object):
454 while indices_to_check and rounds < max_rounds:
455 # TODO: Use retry-after...
456 time.sleep(min_sleep)
457- all_failed_achalls = set()
458+ all_failed_achalls = set() # type: Set[achallenges.KeyAuthorizationAnnotatedChallenge]
459 for index in indices_to_check:
460 comp_achalls, failed_achalls = self._handle_check(
461 aauthzrs, index, chall_update[index])
462@@ -424,7 +426,7 @@ def _find_smart_path(challbs, preferences, combinations):
463
464 # max_cost is now equal to sum(indices) + 1
465
466- best_combo = []
467+ best_combo = None
468 # Set above completing all of the available challenges
469 best_combo_cost = max_cost
470
471@@ -479,7 +481,7 @@ def _report_no_chall_path(challbs):
472 msg += (
473 " You may need to use an authenticator "
474 "plugin that can do challenges over DNS.")
475- logger.fatal(msg)
476+ logger.critical(msg)
477 raise errors.AuthorizationError(msg)
478
479
480@@ -522,11 +524,11 @@ def _report_failed_challs(failed_achalls):
481 :class:`certbot.achallenges.AnnotatedChallenge`.
482
483 """
484- problems = dict()
485+ problems = collections.defaultdict(list)\
486+ # type: DefaultDict[str, List[achallenges.KeyAuthorizationAnnotatedChallenge]]
487 for achall in failed_achalls:
488 if achall.error:
489- problems.setdefault(achall.error.typ, []).append(achall)
490-
491+ problems[achall.error.typ].append(achall)
492 reporter = zope.component.getUtility(interfaces.IReporter)
493 for achalls in six.itervalues(problems):
494 reporter.add_message(
495diff --git a/certbot/cert_manager.py b/certbot/cert_manager.py
496index 4240a05..771ca8c 100644
497--- a/certbot/cert_manager.py
498+++ b/certbot/cert_manager.py
499@@ -7,6 +7,7 @@ import re
500 import traceback
501 import zope.component
502
503+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
504 from certbot import crypto_util
505 from certbot import errors
506 from certbot import interfaces
507@@ -46,7 +47,7 @@ def rename_lineage(config):
508 """
509 disp = zope.component.getUtility(interfaces.IDisplay)
510
511- certname = _get_certnames(config, "rename")[0]
512+ certname = get_certnames(config, "rename")[0]
513
514 new_certname = config.new_certname
515 if not new_certname:
516@@ -88,7 +89,7 @@ def certificates(config):
517
518 def delete(config):
519 """Delete Certbot files associated with a certificate lineage."""
520- certnames = _get_certnames(config, "delete", allow_multiple=True)
521+ certnames = get_certnames(config, "delete", allow_multiple=True)
522 for certname in certnames:
523 storage.delete_files(config, certname)
524 disp = zope.component.getUtility(interfaces.IDisplay)
525@@ -226,7 +227,7 @@ def match_and_check_overlaps(cli_config, acceptable_matches, match_func, rv_func
526 def find_matches(candidate_lineage, return_value, acceptable_matches):
527 """Returns a list of matches using _search_lineages."""
528 acceptable_matches = [func(candidate_lineage) for func in acceptable_matches]
529- acceptable_matches_rv = []
530+ acceptable_matches_rv = [] # type: List[str]
531 for item in acceptable_matches:
532 if isinstance(item, list):
533 acceptable_matches_rv += item
534@@ -288,11 +289,7 @@ def human_readable_cert_info(config, cert, skip_filter_checks=False):
535 cert.privkey))
536 return "".join(certinfo)
537
538-###################
539-# Private Helpers
540-###################
541-
542-def _get_certnames(config, verb, allow_multiple=False):
543+def get_certnames(config, verb, allow_multiple=False, custom_prompt=None):
544 """Get certname from flag, interactively, or error out.
545 """
546 certname = config.certname
547@@ -305,22 +302,32 @@ def _get_certnames(config, verb, allow_multiple=False):
548 if not choices:
549 raise errors.Error("No existing certificates found.")
550 if allow_multiple:
551+ if not custom_prompt:
552+ prompt = "Which certificate(s) would you like to {0}?".format(verb)
553+ else:
554+ prompt = custom_prompt
555 code, certnames = disp.checklist(
556- "Which certificate(s) would you like to {0}?".format(verb),
557- choices, cli_flag="--cert-name",
558- force_interactive=True)
559+ prompt, choices, cli_flag="--cert-name", force_interactive=True)
560 if code != display_util.OK:
561 raise errors.Error("User ended interaction.")
562 else:
563- code, index = disp.menu("Which certificate would you like to {0}?".format(verb),
564- choices, cli_flag="--cert-name",
565- force_interactive=True)
566+ if not custom_prompt:
567+ prompt = "Which certificate would you like to {0}?".format(verb)
568+ else:
569+ prompt = custom_prompt
570+
571+ code, index = disp.menu(
572+ prompt, choices, cli_flag="--cert-name", force_interactive=True)
573
574 if code != display_util.OK or index not in range(0, len(choices)):
575 raise errors.Error("User ended interaction.")
576 certnames = [choices[index]]
577 return certnames
578
579+###################
580+# Private Helpers
581+###################
582+
583 def _report_lines(msgs):
584 """Format a results report for a category of single-line renewal outcomes"""
585 return " " + "\n ".join(str(msg) for msg in msgs)
586@@ -334,7 +341,7 @@ def _report_human_readable(config, parsed_certs):
587
588 def _describe_certs(config, parsed_certs, parse_failures):
589 """Print information about the certs we know about"""
590- out = []
591+ out = [] # type: List[str]
592
593 notify = out.append
594
595@@ -346,7 +353,7 @@ def _describe_certs(config, parsed_certs, parse_failures):
596 notify("Found the following {0}certs:".format(match))
597 notify(_report_human_readable(config, parsed_certs))
598 if parse_failures:
599- notify("\nThe following renewal configuration files "
600+ notify("\nThe following renewal configurations "
601 "were invalid:")
602 notify(_report_lines(parse_failures))
603
604diff --git a/certbot/cli.py b/certbot/cli.py
605index 1c2273c..2c4aa65 100644
606--- a/certbot/cli.py
607+++ b/certbot/cli.py
608@@ -12,10 +12,14 @@ import sys
609 import configargparse
610 import six
611 import zope.component
612+import zope.interface
613
614 from zope.interface import interfaces as zope_interfaces
615
616 from acme import challenges
617+# pylint: disable=unused-import, no-name-in-module
618+from acme.magic_typing import Any, Dict, Optional
619+# pylint: enable=unused-import, no-name-in-module
620
621 import certbot
622
623@@ -28,12 +32,13 @@ from certbot import util
624
625 from certbot.display import util as display_util
626 from certbot.plugins import disco as plugins_disco
627+import certbot.plugins.enhancements as enhancements
628 import certbot.plugins.selection as plugin_selection
629
630 logger = logging.getLogger(__name__)
631
632 # Global, to save us from a lot of argument passing within the scope of this module
633-helpful_parser = None
634+helpful_parser = None # type: Optional[HelpfulArgumentParser]
635
636 # For help strings, figure out how the user ran us.
637 # When invoked from letsencrypt-auto, sys.argv[0] is something like:
638@@ -76,6 +81,7 @@ obtain, install, and renew certificates:
639 (default) run Obtain & install a certificate in your current webserver
640 certonly Obtain or renew a certificate, but do not install it
641 renew Renew all previously obtained certificates that are near expiry
642+ enhance Add security enhancements to your existing configuration
643 -d DOMAINS Comma-separated list of domains to obtain a certificate for
644
645 %s
646@@ -195,17 +201,17 @@ def set_by_cli(var):
647 (CLI or config file) including if the user explicitly set it to the
648 default. Returns False if the variable was assigned a default value.
649 """
650- detector = set_by_cli.detector
651- if detector is None:
652+ detector = set_by_cli.detector # type: ignore
653+ if detector is None and helpful_parser is not None:
654 # Setup on first run: `detector` is a weird version of config in which
655 # the default value of every attribute is wrangled to be boolean-false
656 plugins = plugins_disco.PluginsRegistry.find_all()
657 # reconstructed_args == sys.argv[1:], or whatever was passed to main()
658 reconstructed_args = helpful_parser.args + [helpful_parser.verb]
659- detector = set_by_cli.detector = prepare_and_parse_args(
660+ detector = set_by_cli.detector = prepare_and_parse_args( # type: ignore
661 plugins, reconstructed_args, detect_defaults=True)
662 # propagate plugin requests: eg --standalone modifies config.authenticator
663- detector.authenticator, detector.installer = (
664+ detector.authenticator, detector.installer = ( # type: ignore
665 plugin_selection.cli_plugin_requests(detector))
666
667 if not isinstance(getattr(detector, var), _Default):
668@@ -219,7 +225,10 @@ def set_by_cli(var):
669 return True
670
671 return False
672+
673 # static housekeeping var
674+# functions attributed are not supported by mypy
675+# https://github.com/python/mypy/issues/2087
676 set_by_cli.detector = None # type: ignore
677
678
679@@ -235,8 +244,10 @@ def has_default_value(option, value):
680 :rtype: bool
681
682 """
683- return (option in helpful_parser.defaults and
684- helpful_parser.defaults[option] == value)
685+ if helpful_parser is not None:
686+ return (option in helpful_parser.defaults and
687+ helpful_parser.defaults[option] == value)
688+ return False
689
690
691 def option_was_set(option, value):
692@@ -253,11 +264,12 @@ def option_was_set(option, value):
693
694
695 def argparse_type(variable):
696- "Return our argparse type function for a config variable (default: str)"
697+ """Return our argparse type function for a config variable (default: str)"""
698 # pylint: disable=protected-access
699- for action in helpful_parser.parser._actions:
700- if action.type is not None and action.dest == variable:
701- return action.type
702+ if helpful_parser is not None:
703+ for action in helpful_parser.parser._actions:
704+ if action.type is not None and action.dest == variable:
705+ return action.type
706 return str
707
708 def read_file(filename, mode="rb"):
709@@ -290,10 +302,12 @@ def flag_default(name):
710
711 def config_help(name, hidden=False):
712 """Extract the help message for an `.IConfig` attribute."""
713+ # pylint: disable=no-member
714 if hidden:
715 return argparse.SUPPRESS
716 else:
717- return interfaces.IConfig[name].__doc__
718+ field = interfaces.IConfig.__getitem__(name) # type: zope.interface.interface.Attribute
719+ return field.__doc__
720
721
722 class HelpfulArgumentGroup(object):
723@@ -415,6 +429,12 @@ VERB_HELP = [
724 os.path.join(flag_default("config_dir"), "live"))),
725 "usage": "\n\n certbot update_symlinks [options]\n\n"
726 }),
727+ ("enhance", {
728+ "short": "Add security enhancements to your existing configuration",
729+ "opts": ("Helps to harden the TLS configuration by adding security enhancements "
730+ "to already existing configuration."),
731+ "usage": "\n\n certbot enhance [options]\n\n"
732+ }),
733
734 ]
735 # VERB_HELP is a list in order to preserve order, but a dict is sometimes useful
736@@ -449,6 +469,7 @@ class HelpfulArgumentParser(object):
737 "update_symlinks": main.update_symlinks,
738 "certificates": main.certificates,
739 "delete": main.delete,
740+ "enhance": main.enhance,
741 }
742
743 # Get notification function for printing
744@@ -465,7 +486,7 @@ class HelpfulArgumentParser(object):
745 HELP_TOPICS += list(self.VERBS) + self.COMMANDS_TOPICS + ["manage"]
746
747 plugin_names = list(plugins)
748- self.help_topics = HELP_TOPICS + plugin_names + [None]
749+ self.help_topics = HELP_TOPICS + plugin_names + [None] # type: ignore
750
751 self.detect_defaults = detect_defaults
752 self.args = args
753@@ -484,8 +505,11 @@ class HelpfulArgumentParser(object):
754 short_usage = self._usage_string(plugins, self.help_arg)
755
756 self.visible_topics = self.determine_help_topics(self.help_arg)
757- self.groups = {} # elements are added by .add_group()
758- self.defaults = {} # elements are added by .parse_args()
759+
760+ # elements are added by .add_group()
761+ self.groups = {} # type: Dict[str, argparse._ArgumentGroup]
762+ # elements are added by .parse_args()
763+ self.defaults = {} # type: Dict[str, Any]
764
765 self.parser = configargparse.ArgParser(
766 prog="certbot",
767@@ -604,6 +628,10 @@ class HelpfulArgumentParser(object):
768 raise errors.Error("Using --allow-subset-of-names with a"
769 " wildcard domain is not supported.")
770
771+ if parsed_args.hsts and parsed_args.auto_hsts:
772+ raise errors.Error(
773+ "Parameters --hsts and --auto-hsts cannot be used simultaneously.")
774+
775 possible_deprecation_warning(parsed_args)
776
777 return parsed_args
778@@ -797,7 +825,6 @@ class HelpfulArgumentParser(object):
779 if self.help_arg:
780 for v in verbs:
781 self.groups[topic].add_argument(v, help=VERB_HELP_MAP[v]["short"])
782-
783 return HelpfulArgumentGroup(self, topic)
784
785 def add_plugin_args(self, plugins):
786@@ -883,21 +910,22 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
787 "flag to 0 disables log rotation entirely, causing "
788 "Certbot to always append to the same log file.")
789 helpful.add(
790- [None, "automation", "run", "certonly"], "-n", "--non-interactive", "--noninteractive",
791+ [None, "automation", "run", "certonly", "enhance"],
792+ "-n", "--non-interactive", "--noninteractive",
793 dest="noninteractive_mode", action="store_true",
794 default=flag_default("noninteractive_mode"),
795 help="Run without ever asking for user input. This may require "
796 "additional command line flags; the client will try to explain "
797 "which ones are required if it finds one missing")
798 helpful.add(
799- [None, "register", "run", "certonly"],
800+ [None, "register", "run", "certonly", "enhance"],
801 constants.FORCE_INTERACTIVE_FLAG, action="store_true",
802 default=flag_default("force_interactive"),
803 help="Force Certbot to be interactive even if it detects it's not "
804 "being run in a terminal. This flag cannot be used with the "
805 "renew subcommand.")
806 helpful.add(
807- [None, "run", "certonly", "certificates"],
808+ [None, "run", "certonly", "certificates", "enhance"],
809 "-d", "--domains", "--domain", dest="domains",
810 metavar="DOMAIN", action=_DomainsAction,
811 default=flag_default("domains"),
812@@ -913,8 +941,8 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
813 "name. In the case of a name collision it will append a number "
814 "like 0001 to the file path name. (default: Ask)")
815 helpful.add(
816- [None, "run", "certonly", "manage", "delete", "certificates", "renew"],
817- "--cert-name", dest="certname",
818+ [None, "run", "certonly", "manage", "delete", "certificates",
819+ "renew", "enhance"], "--cert-name", dest="certname",
820 metavar="CERTNAME", default=flag_default("certname"),
821 help="Certificate name to apply. This name is used by Certbot for housekeeping "
822 "and in file paths; it doesn't affect the content of the certificate itself. "
823@@ -995,6 +1023,12 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
824 "but does not match the requested domains, renew it now, "
825 "regardless of whether it is near expiry.")
826 helpful.add(
827+ "automation", "--reuse-key", dest="reuse_key",
828+ action="store_true", default=flag_default("reuse_key"),
829+ help="When renewing, use the same private key as the existing "
830+ "certificate.")
831+
832+ helpful.add(
833 ["automation", "renew", "certonly"],
834 "--allow-subset-of-names", action="store_true",
835 default=flag_default("allow_subset_of_names"),
836@@ -1048,7 +1082,7 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
837 help="Show tracebacks in case of errors, and allow certbot-auto "
838 "execution on experimental platforms")
839 helpful.add(
840- [None, "certonly", "renew", "run"], "--debug-challenges", action="store_true",
841+ [None, "certonly", "run"], "--debug-challenges", action="store_true",
842 default=flag_default("debug_challenges"),
843 help="After setting up challenges, wait for user input before "
844 "submitting to CA")
845@@ -1085,7 +1119,8 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
846 dest="must_staple", default=flag_default("must_staple"),
847 help=config_help("must_staple"))
848 helpful.add(
849- "security", "--redirect", action="store_true", dest="redirect",
850+ ["security", "enhance"],
851+ "--redirect", action="store_true", dest="redirect",
852 default=flag_default("redirect"),
853 help="Automatically redirect all HTTP traffic to HTTPS for the newly "
854 "authenticated vhost. (default: Ask)")
855@@ -1095,7 +1130,8 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
856 help="Do not automatically redirect all HTTP traffic to HTTPS for the newly "
857 "authenticated vhost. (default: Ask)")
858 helpful.add(
859- "security", "--hsts", action="store_true", dest="hsts", default=flag_default("hsts"),
860+ ["security", "enhance"],
861+ "--hsts", action="store_true", dest="hsts", default=flag_default("hsts"),
862 help="Add the Strict-Transport-Security header to every HTTP response."
863 " Forcing browser to always use SSL for the domain."
864 " Defends against SSL Stripping.")
865@@ -1103,7 +1139,8 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
866 "security", "--no-hsts", action="store_false", dest="hsts",
867 default=flag_default("hsts"), help=argparse.SUPPRESS)
868 helpful.add(
869- "security", "--uir", action="store_true", dest="uir", default=flag_default("uir"),
870+ ["security", "enhance"],
871+ "--uir", action="store_true", dest="uir", default=flag_default("uir"),
872 help='Add the "Content-Security-Policy: upgrade-insecure-requests"'
873 ' header to every HTTP response. Forcing the browser to use'
874 ' https:// for every http:// resource.')
875@@ -1180,10 +1217,25 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis
876 default=flag_default("directory_hooks"), dest="directory_hooks",
877 help="Disable running executables found in Certbot's hook directories"
878 " during renewal. (default: False)")
879+ helpful.add(
880+ "renew", "--disable-renew-updates", action="store_true",
881+ default=flag_default("disable_renew_updates"), dest="disable_renew_updates",
882+ help="Disable automatic updates to your server configuration that"
883+ " would otherwise be done by the selected installer plugin, and triggered"
884+ " when the user executes \"certbot renew\", regardless of if the certificate"
885+ " is renewed. This setting does not apply to important TLS configuration"
886+ " updates.")
887+ helpful.add(
888+ "renew", "--no-autorenew", action="store_false",
889+ default=flag_default("autorenew"), dest="autorenew",
890+ help="Disable auto renewal of certificates.")
891
892 helpful.add_deprecated_argument("--agree-dev-preview", 0)
893 helpful.add_deprecated_argument("--dialog", 0)
894
895+ # Populate the command line parameters for new style enhancements
896+ enhancements.populate_cli(helpful.add)
897+
898 _create_subparsers(helpful)
899 _paths_parser(helpful)
900 # _plugins_parsing should be the last thing to act upon the main
901@@ -1276,14 +1328,14 @@ def _paths_parser(helpful):
902 verb = helpful.help_arg
903
904 cph = "Path to where certificate is saved (with auth --csr), installed from, or revoked."
905- section = ["paths", "install", "revoke", "certonly", "manage"]
906+ sections = ["paths", "install", "revoke", "certonly", "manage"]
907 if verb == "certonly":
908- add(section, "--cert-path", type=os.path.abspath,
909+ add(sections, "--cert-path", type=os.path.abspath,
910 default=flag_default("auth_cert_path"), help=cph)
911 elif verb == "revoke":
912- add(section, "--cert-path", type=read_file, required=True, help=cph)
913+ add(sections, "--cert-path", type=read_file, required=True, help=cph)
914 else:
915- add(section, "--cert-path", type=os.path.abspath, help=cph)
916+ add(sections, "--cert-path", type=os.path.abspath, help=cph)
917
918 section = "paths"
919 if verb in ("install", "revoke"):
920@@ -1363,10 +1415,18 @@ def _plugins_parsing(helpful, plugins):
921 default=flag_default("dns_dnsmadeeasy"),
922 help=("Obtain certificates using a DNS TXT record (if you are"
923 "using DNS Made Easy for DNS)."))
924+ helpful.add(["plugins", "certonly"], "--dns-gehirn", action="store_true",
925+ default=flag_default("dns_gehirn"),
926+ help=("Obtain certificates using a DNS TXT record "
927+ "(if you are using Gehirn Infrastracture Service for DNS)."))
928 helpful.add(["plugins", "certonly"], "--dns-google", action="store_true",
929 default=flag_default("dns_google"),
930 help=("Obtain certificates using a DNS TXT record (if you are "
931 "using Google Cloud DNS)."))
932+ helpful.add(["plugins", "certonly"], "--dns-linode", action="store_true",
933+ default=flag_default("dns_linode"),
934+ help=("Obtain certificates using a DNS TXT record (if you are "
935+ "using Linode for DNS)."))
936 helpful.add(["plugins", "certonly"], "--dns-luadns", action="store_true",
937 default=flag_default("dns_luadns"),
938 help=("Obtain certificates using a DNS TXT record (if you are "
939@@ -1375,6 +1435,10 @@ def _plugins_parsing(helpful, plugins):
940 default=flag_default("dns_nsone"),
941 help=("Obtain certificates using a DNS TXT record (if you are "
942 "using NS1 for DNS)."))
943+ helpful.add(["plugins", "certonly"], "--dns-ovh", action="store_true",
944+ default=flag_default("dns_ovh"),
945+ help=("Obtain certificates using a DNS TXT record (if you are "
946+ "using OVH for DNS)."))
947 helpful.add(["plugins", "certonly"], "--dns-rfc2136", action="store_true",
948 default=flag_default("dns_rfc2136"),
949 help="Obtain certificates using a DNS TXT record (if you are using BIND for DNS).")
950@@ -1382,6 +1446,10 @@ def _plugins_parsing(helpful, plugins):
951 default=flag_default("dns_route53"),
952 help=("Obtain certificates using a DNS TXT record (if you are using Route53 for "
953 "DNS)."))
954+ helpful.add(["plugins", "certonly"], "--dns-sakuracloud", action="store_true",
955+ default=flag_default("dns_sakuracloud"),
956+ help=("Obtain certificates using a DNS TXT record "
957+ "(if you are using Sakura Cloud for DNS)."))
958
959 # things should not be reorder past/pre this comment:
960 # plugins_group should be displayed in --help before plugin
961diff --git a/certbot/client.py b/certbot/client.py
962index 2992c0c..4d4915a 100644
963--- a/certbot/client.py
964+++ b/certbot/client.py
965@@ -4,8 +4,11 @@ import logging
966 import os
967 import platform
968
969+
970 from cryptography.hazmat.backends import default_backend
971-from cryptography.hazmat.primitives.asymmetric import rsa
972+# https://github.com/python/typeshed/blob/master/third_party/
973+# 2/cryptography/hazmat/primitives/asymmetric/rsa.pyi
974+from cryptography.hazmat.primitives.asymmetric.rsa import generate_private_key # type: ignore
975 import josepy as jose
976 import OpenSSL
977 import zope.component
978@@ -14,6 +17,7 @@ from acme import client as acme_client
979 from acme import crypto_util as acme_crypto_util
980 from acme import errors as acme_errors
981 from acme import messages
982+from acme.magic_typing import Optional # pylint: disable=unused-import,no-name-in-module
983
984 import certbot
985
986@@ -61,9 +65,17 @@ def determine_user_agent(config):
987 if config.user_agent is None:
988 ua = ("CertbotACMEClient/{0} ({1}; {2}{8}) Authenticator/{3} Installer/{4} "
989 "({5}; flags: {6}) Py/{7}")
990- ua = ua.format(certbot.__version__, cli.cli_command, util.get_os_info_ua(),
991+ if os.environ.get("CERTBOT_DOCS") == "1":
992+ cli_command = "certbot(-auto)"
993+ os_info = "OS_NAME OS_VERSION"
994+ python_version = "major.minor.patchlevel"
995+ else:
996+ cli_command = cli.cli_command
997+ os_info = util.get_os_info_ua()
998+ python_version = platform.python_version()
999+ ua = ua.format(certbot.__version__, cli_command, os_info,
1000 config.authenticator, config.installer, config.verb,
1001- ua_flags(config), platform.python_version(),
1002+ ua_flags(config), python_version,
1003 "; " + config.user_agent_comment if config.user_agent_comment else "")
1004 else:
1005 ua = config.user_agent
1006@@ -155,12 +167,16 @@ def register(config, account_storage, tos_cb=None):
1007 if not config.dry_run:
1008 logger.info("Registering without email!")
1009
1010+ # If --dry-run is used, and there is no staging account, create one with no email.
1011+ if config.dry_run:
1012+ config.email = None
1013+
1014 # Each new registration shall use a fresh new key
1015- key = jose.JWKRSA(key=jose.ComparableRSAKey(
1016- rsa.generate_private_key(
1017+ rsa_key = generate_private_key(
1018 public_exponent=65537,
1019 key_size=config.rsa_key_size,
1020- backend=default_backend())))
1021+ backend=default_backend())
1022+ key = jose.JWKRSA(key=jose.ComparableRSAKey(rsa_key))
1023 acme = acme_from_config_key(config, key)
1024 # TODO: add phone?
1025 regr = perform_registration(acme, config, tos_cb)
1026@@ -179,8 +195,9 @@ def perform_registration(acme, config, tos_cb):
1027 Actually register new account, trying repeatedly if there are email
1028 problems
1029
1030- :param .IConfig config: Client configuration.
1031 :param acme.client.Client client: ACME client object.
1032+ :param .IConfig config: Client configuration.
1033+ :param Callable tos_cb: a callback to handle Term of Service agreement.
1034
1035 :returns: Registration Resource.
1036 :rtype: `acme.messages.RegistrationResource`
1037@@ -266,7 +283,7 @@ class Client(object):
1038 cert, chain = crypto_util.cert_and_chain_from_fullchain(orderr.fullchain_pem)
1039 return cert.encode(), chain.encode()
1040
1041- def obtain_certificate(self, domains):
1042+ def obtain_certificate(self, domains, old_keypath=None):
1043 """Obtains a certificate from the ACME server.
1044
1045 `.register` must be called before `.obtain_certificate`
1046@@ -279,16 +296,39 @@ class Client(object):
1047 :rtype: tuple
1048
1049 """
1050+
1051+ # We need to determine the key path, key PEM data, CSR path,
1052+ # and CSR PEM data. For a dry run, the paths are None because
1053+ # they aren't permanently saved to disk. For a lineage with
1054+ # --reuse-key, the key path and PEM data are derived from an
1055+ # existing file.
1056+
1057+ if old_keypath is not None:
1058+ # We've been asked to reuse a specific existing private key.
1059+ # Therefore, we'll read it now and not generate a new one in
1060+ # either case below.
1061+ #
1062+ # We read in bytes here because the type of `key.pem`
1063+ # created below is also bytes.
1064+ with open(old_keypath, "rb") as f:
1065+ keypath = old_keypath
1066+ keypem = f.read()
1067+ key = util.Key(file=keypath, pem=keypem) # type: Optional[util.Key]
1068+ logger.info("Reusing existing private key from %s.", old_keypath)
1069+ else:
1070+ # The key is set to None here but will be created below.
1071+ key = None
1072+
1073 # Create CSR from names
1074 if self.config.dry_run:
1075- key = util.Key(file=None,
1076- pem=crypto_util.make_key(self.config.rsa_key_size))
1077+ key = key or util.Key(file=None,
1078+ pem=crypto_util.make_key(self.config.rsa_key_size))
1079 csr = util.CSR(file=None, form="pem",
1080 data=acme_crypto_util.make_csr(
1081 key.pem, domains, self.config.must_staple))
1082 else:
1083- key = crypto_util.init_save_key(
1084- self.config.rsa_key_size, self.config.key_dir)
1085+ key = key or crypto_util.init_save_key(self.config.rsa_key_size,
1086+ self.config.key_dir)
1087 csr = crypto_util.init_save_csr(key, domains, self.config.csr_dir)
1088
1089 orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
1090@@ -338,9 +378,10 @@ class Client(object):
1091 authenticator and installer, and then create a new renewable lineage
1092 containing it.
1093
1094- :param list domains: Domains to request.
1095- :param plugins: A PluginsFactory object.
1096- :param str certname: Name of new cert
1097+ :param domains: domains to request a certificate for
1098+ :type domains: `list` of `str`
1099+ :param certname: requested name of lineage
1100+ :type certname: `str` or `None`
1101
1102 :returns: A new :class:`certbot.storage.RenewableCert` instance
1103 referred to the enrolled cert lineage, False if the cert could not
1104@@ -351,17 +392,11 @@ class Client(object):
1105
1106 if (self.config.config_dir != constants.CLI_DEFAULTS["config_dir"] or
1107 self.config.work_dir != constants.CLI_DEFAULTS["work_dir"]):
1108- logger.warning(
1109+ logger.info(
1110 "Non-standard path(s), might not work with crontab installed "
1111 "by your operating system package manager")
1112
1113- if certname:
1114- new_name = certname
1115- elif util.is_wildcard_domain(domains[0]):
1116- # Don't make files and directories starting with *.
1117- new_name = domains[0][2:]
1118- else:
1119- new_name = domains[0]
1120+ new_name = self._choose_lineagename(domains, certname)
1121
1122 if self.config.dry_run:
1123 logger.debug("Dry run: Skipping creating new lineage for %s",
1124@@ -373,6 +408,26 @@ class Client(object):
1125 key.pem, chain,
1126 self.config)
1127
1128+ def _choose_lineagename(self, domains, certname):
1129+ """Chooses a name for the new lineage.
1130+
1131+ :param domains: domains in certificate request
1132+ :type domains: `list` of `str`
1133+ :param certname: requested name of lineage
1134+ :type certname: `str` or `None`
1135+
1136+ :returns: lineage name that should be used
1137+ :rtype: str
1138+
1139+ """
1140+ if certname:
1141+ return certname
1142+ elif util.is_wildcard_domain(domains[0]):
1143+ # Don't make files and directories starting with *.
1144+ return domains[0][2:]
1145+ else:
1146+ return domains[0]
1147+
1148 def save_certificate(self, cert_pem, chain_pem,
1149 cert_path, chain_path, fullchain_path):
1150 """Saves the certificate received from the ACME server.
1151@@ -451,7 +506,7 @@ class Client(object):
1152 # sites may have been enabled / final cleanup
1153 self.installer.restart()
1154
1155- def enhance_config(self, domains, chain_path):
1156+ def enhance_config(self, domains, chain_path, ask_redirect=True):
1157 """Enhance the configuration.
1158
1159 :param list domains: list of domains to configure
1160@@ -478,8 +533,9 @@ class Client(object):
1161 for config_name, enhancement_name, option in enhancement_info:
1162 config_value = getattr(self.config, config_name)
1163 if enhancement_name in supported:
1164- if config_name == "redirect" and config_value is None:
1165- config_value = enhancements.ask(enhancement_name)
1166+ if ask_redirect:
1167+ if config_name == "redirect" and config_value is None:
1168+ config_value = enhancements.ask(enhancement_name)
1169 if config_value:
1170 self.apply_enhancement(domains, enhancement_name, option)
1171 enhanced = True
1172@@ -515,8 +571,12 @@ class Client(object):
1173 try:
1174 self.installer.enhance(dom, enhancement, options)
1175 except errors.PluginEnhancementAlreadyPresent:
1176- logger.warning("Enhancement %s was already set.",
1177- enhancement)
1178+ if enhancement == "ensure-http-header":
1179+ logger.warning("Enhancement %s was already set.",
1180+ options)
1181+ else:
1182+ logger.warning("Enhancement %s was already set.",
1183+ enhancement)
1184 except errors.PluginError:
1185 logger.warning("Unable to set enhancement %s for %s",
1186 enhancement, dom)
1187@@ -585,8 +645,10 @@ def validate_key_csr(privkey, csr=None):
1188 if csr.form == "der":
1189 csr_obj = OpenSSL.crypto.load_certificate_request(
1190 OpenSSL.crypto.FILETYPE_ASN1, csr.data)
1191- csr = util.CSR(csr.file, OpenSSL.crypto.dump_certificate(
1192- OpenSSL.crypto.FILETYPE_PEM, csr_obj), "pem")
1193+ cert_buffer = OpenSSL.crypto.dump_certificate_request(
1194+ OpenSSL.crypto.FILETYPE_PEM, csr_obj
1195+ )
1196+ csr = util.CSR(csr.file, cert_buffer, "pem")
1197
1198 # If CSR is provided, it must be readable and valid.
1199 if csr.data and not crypto_util.valid_csr(csr.data):
1200diff --git a/certbot/configuration.py b/certbot/configuration.py
1201index 2977956..daf514b 100644
1202--- a/certbot/configuration.py
1203+++ b/certbot/configuration.py
1204@@ -65,8 +65,12 @@ class NamespaceConfig(object):
1205
1206 @property
1207 def accounts_dir(self): # pylint: disable=missing-docstring
1208+ return self.accounts_dir_for_server_path(self.server_path)
1209+
1210+ def accounts_dir_for_server_path(self, server_path):
1211+ """Path to accounts directory based on server_path"""
1212 return os.path.join(
1213- self.namespace.config_dir, constants.ACCOUNTS_DIR, self.server_path)
1214+ self.namespace.config_dir, constants.ACCOUNTS_DIR, server_path)
1215
1216 @property
1217 def backup_dir(self): # pylint: disable=missing-docstring
1218diff --git a/certbot/constants.py b/certbot/constants.py
1219index 0d0ee8d..a2de2d2 100644
1220--- a/certbot/constants.py
1221+++ b/certbot/constants.py
1222@@ -37,6 +37,7 @@ CLI_DEFAULTS = dict(
1223 expand=False,
1224 renew_by_default=False,
1225 renew_with_new_domains=False,
1226+ autorenew=True,
1227 allow_subset_of_names=False,
1228 tos=False,
1229 account=None,
1230@@ -57,6 +58,7 @@ CLI_DEFAULTS = dict(
1231 rsa_key_size=2048,
1232 must_staple=False,
1233 redirect=None,
1234+ auto_hsts=False,
1235 hsts=None,
1236 uir=None,
1237 staple=None,
1238@@ -64,6 +66,8 @@ CLI_DEFAULTS = dict(
1239 pref_challs=[],
1240 validate_hooks=True,
1241 directory_hooks=True,
1242+ reuse_key=False,
1243+ disable_renew_updates=False,
1244
1245 # Subparsers
1246 num=None,
1247@@ -84,7 +88,7 @@ CLI_DEFAULTS = dict(
1248 config_dir="/etc/letsencrypt",
1249 work_dir="/var/lib/letsencrypt",
1250 logs_dir="/var/log/letsencrypt",
1251- server="https://acme-v01.api.letsencrypt.org/directory",
1252+ server="https://acme-v02.api.letsencrypt.org/directory",
1253
1254 # Plugins parsers
1255 configurator=None,
1256@@ -100,11 +104,15 @@ CLI_DEFAULTS = dict(
1257 dns_digitalocean=False,
1258 dns_dnsimple=False,
1259 dns_dnsmadeeasy=False,
1260+ dns_gehirn=False,
1261 dns_google=False,
1262+ dns_linode=False,
1263 dns_luadns=False,
1264 dns_nsone=False,
1265+ dns_ovh=False,
1266 dns_rfc2136=False,
1267- dns_route53=False
1268+ dns_route53=False,
1269+ dns_sakuracloud=False
1270
1271 )
1272 STAGING_URI = "https://acme-staging-v02.api.letsencrypt.org/directory"
1273@@ -156,6 +164,13 @@ CONFIG_DIRS_MODE = 0o755
1274 ACCOUNTS_DIR = "accounts"
1275 """Directory where all accounts are saved."""
1276
1277+LE_REUSE_SERVERS = {
1278+ 'acme-v02.api.letsencrypt.org/directory': 'acme-v01.api.letsencrypt.org/directory',
1279+ 'acme-staging-v02.api.letsencrypt.org/directory':
1280+ 'acme-staging.api.letsencrypt.org/directory'
1281+}
1282+"""Servers that can reuse accounts from other servers."""
1283+
1284 BACKUP_DIR = "backups"
1285 """Directory (relative to `IConfig.work_dir`) where backups are kept."""
1286
1287diff --git a/certbot/crypto_util.py b/certbot/crypto_util.py
1288index bd4e7fc..943e2c8 100644
1289--- a/certbot/crypto_util.py
1290+++ b/certbot/crypto_util.py
1291@@ -7,16 +7,24 @@
1292 import hashlib
1293 import logging
1294 import os
1295+import warnings
1296
1297-import OpenSSL
1298 import pyrfc3339
1299 import six
1300 import zope.component
1301+from cryptography.exceptions import InvalidSignature
1302 from cryptography.hazmat.backends import default_backend
1303-from cryptography import x509
1304+from cryptography.hazmat.primitives.asymmetric.ec import ECDSA
1305+from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePublicKey
1306+from cryptography.hazmat.primitives.asymmetric.padding import PKCS1v15
1307+from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicKey
1308+# https://github.com/python/typeshed/tree/master/third_party/2/cryptography
1309+from cryptography import x509 # type: ignore
1310+from OpenSSL import crypto
1311+from OpenSSL import SSL # type: ignore
1312
1313 from acme import crypto_util as acme_crypto_util
1314-
1315+from acme.magic_typing import IO # pylint: disable=unused-import, no-name-in-module
1316 from certbot import errors
1317 from certbot import interfaces
1318 from certbot import util
1319@@ -47,7 +55,7 @@ def init_save_key(key_size, key_dir, keyname="key-certbot.pem"):
1320 try:
1321 key_pem = make_key(key_size)
1322 except ValueError as err:
1323- logger.exception(err)
1324+ logger.error("", exc_info=True)
1325 raise err
1326
1327 config = zope.component.getUtility(interfaces.IConfig)
1328@@ -111,11 +119,11 @@ def valid_csr(csr):
1329
1330 """
1331 try:
1332- req = OpenSSL.crypto.load_certificate_request(
1333- OpenSSL.crypto.FILETYPE_PEM, csr)
1334+ req = crypto.load_certificate_request(
1335+ crypto.FILETYPE_PEM, csr)
1336 return req.verify(req.get_pubkey())
1337- except OpenSSL.crypto.Error as error:
1338- logger.debug(error, exc_info=True)
1339+ except crypto.Error:
1340+ logger.debug("", exc_info=True)
1341 return False
1342
1343
1344@@ -129,13 +137,13 @@ def csr_matches_pubkey(csr, privkey):
1345 :rtype: bool
1346
1347 """
1348- req = OpenSSL.crypto.load_certificate_request(
1349- OpenSSL.crypto.FILETYPE_PEM, csr)
1350- pkey = OpenSSL.crypto.load_privatekey(OpenSSL.crypto.FILETYPE_PEM, privkey)
1351+ req = crypto.load_certificate_request(
1352+ crypto.FILETYPE_PEM, csr)
1353+ pkey = crypto.load_privatekey(crypto.FILETYPE_PEM, privkey)
1354 try:
1355 return req.verify(pkey)
1356- except OpenSSL.crypto.Error as error:
1357- logger.debug(error, exc_info=True)
1358+ except crypto.Error:
1359+ logger.debug("", exc_info=True)
1360 return False
1361
1362
1363@@ -145,26 +153,26 @@ def import_csr_file(csrfile, data):
1364 :param str csrfile: CSR filename
1365 :param str data: contents of the CSR file
1366
1367- :returns: (`OpenSSL.crypto.FILETYPE_PEM`,
1368+ :returns: (`crypto.FILETYPE_PEM`,
1369 util.CSR object representing the CSR,
1370 list of domains requested in the CSR)
1371 :rtype: tuple
1372
1373 """
1374- PEM = OpenSSL.crypto.FILETYPE_PEM
1375- load = OpenSSL.crypto.load_certificate_request
1376+ PEM = crypto.FILETYPE_PEM
1377+ load = crypto.load_certificate_request
1378 try:
1379 # Try to parse as DER first, then fall back to PEM.
1380- csr = load(OpenSSL.crypto.FILETYPE_ASN1, data)
1381- except OpenSSL.crypto.Error:
1382+ csr = load(crypto.FILETYPE_ASN1, data)
1383+ except crypto.Error:
1384 try:
1385 csr = load(PEM, data)
1386- except OpenSSL.crypto.Error:
1387+ except crypto.Error:
1388 raise errors.Error("Failed to parse CSR file: {0}".format(csrfile))
1389
1390 domains = _get_names_from_loaded_cert_or_req(csr)
1391 # Internally we always use PEM, so re-encode as PEM before returning.
1392- data_pem = OpenSSL.crypto.dump_certificate_request(PEM, csr)
1393+ data_pem = crypto.dump_certificate_request(PEM, csr)
1394 return PEM, util.CSR(file=csrfile, data=data_pem, form="pem"), domains
1395
1396
1397@@ -178,9 +186,9 @@ def make_key(bits):
1398
1399 """
1400 assert bits >= 1024 # XXX
1401- key = OpenSSL.crypto.PKey()
1402- key.generate_key(OpenSSL.crypto.TYPE_RSA, bits)
1403- return OpenSSL.crypto.dump_privatekey(OpenSSL.crypto.FILETYPE_PEM, key)
1404+ key = crypto.PKey()
1405+ key.generate_key(crypto.TYPE_RSA, bits)
1406+ return crypto.dump_privatekey(crypto.FILETYPE_PEM, key)
1407
1408
1409 def valid_privkey(privkey):
1410@@ -193,9 +201,9 @@ def valid_privkey(privkey):
1411
1412 """
1413 try:
1414- return OpenSSL.crypto.load_privatekey(
1415- OpenSSL.crypto.FILETYPE_PEM, privkey).check()
1416- except (TypeError, OpenSSL.crypto.Error):
1417+ return crypto.load_privatekey(
1418+ crypto.FILETYPE_PEM, privkey).check()
1419+ except (TypeError, crypto.Error):
1420 return False
1421
1422
1423@@ -224,13 +232,29 @@ def verify_renewable_cert_sig(renewable_cert):
1424 :raises errors.Error: If signature verification fails.
1425 """
1426 try:
1427- with open(renewable_cert.chain, 'rb') as chain:
1428- chain, _ = pyopenssl_load_certificate(chain.read())
1429- with open(renewable_cert.cert, 'rb') as cert:
1430- cert = x509.load_pem_x509_certificate(cert.read(), default_backend())
1431- hash_name = cert.signature_hash_algorithm.name
1432- OpenSSL.crypto.verify(chain, cert.signature, cert.tbs_certificate_bytes, hash_name)
1433- except (IOError, ValueError, OpenSSL.crypto.Error) as e:
1434+ with open(renewable_cert.chain, 'rb') as chain_file: # type: IO[bytes]
1435+ chain = x509.load_pem_x509_certificate(chain_file.read(), default_backend())
1436+ with open(renewable_cert.cert, 'rb') as cert_file: # type: IO[bytes]
1437+ cert = x509.load_pem_x509_certificate(cert_file.read(), default_backend())
1438+ pk = chain.public_key()
1439+ with warnings.catch_warnings():
1440+ warnings.simplefilter("ignore")
1441+ if isinstance(pk, RSAPublicKey):
1442+ # https://github.com/python/typeshed/blob/master/third_party/2/cryptography/hazmat/primitives/asymmetric/rsa.pyi
1443+ verifier = pk.verifier( # type: ignore
1444+ cert.signature, PKCS1v15(), cert.signature_hash_algorithm
1445+ )
1446+ verifier.update(cert.tbs_certificate_bytes)
1447+ verifier.verify()
1448+ elif isinstance(pk, EllipticCurvePublicKey):
1449+ verifier = pk.verifier(
1450+ cert.signature, ECDSA(cert.signature_hash_algorithm)
1451+ )
1452+ verifier.update(cert.tbs_certificate_bytes)
1453+ verifier.verify()
1454+ else:
1455+ raise errors.Error("Unsupported public key type")
1456+ except (IOError, ValueError, InvalidSignature) as e:
1457 error_str = "verifying the signature of the cert located at {0} has failed. \
1458 Details: {1}".format(renewable_cert.cert, e)
1459 logger.exception(error_str)
1460@@ -246,11 +270,11 @@ def verify_cert_matches_priv_key(cert_path, key_path):
1461 :raises errors.Error: If they don't match.
1462 """
1463 try:
1464- context = OpenSSL.SSL.Context(OpenSSL.SSL.SSLv23_METHOD)
1465+ context = SSL.Context(SSL.SSLv23_METHOD)
1466 context.use_certificate_file(cert_path)
1467 context.use_privatekey_file(key_path)
1468 context.check_privatekey()
1469- except (IOError, OpenSSL.SSL.Error) as e:
1470+ except (IOError, SSL.Error) as e:
1471 error_str = "verifying the cert located at {0} matches the \
1472 private key located at {1} has failed. \
1473 Details: {2}".format(cert_path,
1474@@ -267,12 +291,12 @@ def verify_fullchain(renewable_cert):
1475 :raises errors.Error: If cert and chain do not combine to fullchain.
1476 """
1477 try:
1478- with open(renewable_cert.chain) as chain:
1479- chain = chain.read()
1480- with open(renewable_cert.cert) as cert:
1481- cert = cert.read()
1482- with open(renewable_cert.fullchain) as fullchain:
1483- fullchain = fullchain.read()
1484+ with open(renewable_cert.chain) as chain_file: # type: IO[str]
1485+ chain = chain_file.read()
1486+ with open(renewable_cert.cert) as cert_file: # type: IO[str]
1487+ cert = cert_file.read()
1488+ with open(renewable_cert.fullchain) as fullchain_file: # type: IO[str]
1489+ fullchain = fullchain_file.read()
1490 if (cert + chain) != fullchain:
1491 error_str = "fullchain does not match cert + chain for {0}!"
1492 error_str = error_str.format(renewable_cert.lineagename)
1493@@ -294,43 +318,43 @@ def pyopenssl_load_certificate(data):
1494
1495 openssl_errors = []
1496
1497- for file_type in (OpenSSL.crypto.FILETYPE_PEM, OpenSSL.crypto.FILETYPE_ASN1):
1498+ for file_type in (crypto.FILETYPE_PEM, crypto.FILETYPE_ASN1):
1499 try:
1500- return OpenSSL.crypto.load_certificate(file_type, data), file_type
1501- except OpenSSL.crypto.Error as error: # TODO: other errors?
1502+ return crypto.load_certificate(file_type, data), file_type
1503+ except crypto.Error as error: # TODO: other errors?
1504 openssl_errors.append(error)
1505 raise errors.Error("Unable to load: {0}".format(",".join(
1506 str(error) for error in openssl_errors)))
1507
1508
1509 def _load_cert_or_req(cert_or_req_str, load_func,
1510- typ=OpenSSL.crypto.FILETYPE_PEM):
1511+ typ=crypto.FILETYPE_PEM):
1512 try:
1513 return load_func(typ, cert_or_req_str)
1514- except OpenSSL.crypto.Error as error:
1515- logger.exception(error)
1516+ except crypto.Error:
1517+ logger.error("", exc_info=True)
1518 raise
1519
1520
1521 def _get_sans_from_cert_or_req(cert_or_req_str, load_func,
1522- typ=OpenSSL.crypto.FILETYPE_PEM):
1523+ typ=crypto.FILETYPE_PEM):
1524 # pylint: disable=protected-access
1525 return acme_crypto_util._pyopenssl_cert_or_req_san(_load_cert_or_req(
1526 cert_or_req_str, load_func, typ))
1527
1528
1529-def get_sans_from_cert(cert, typ=OpenSSL.crypto.FILETYPE_PEM):
1530+def get_sans_from_cert(cert, typ=crypto.FILETYPE_PEM):
1531 """Get a list of Subject Alternative Names from a certificate.
1532
1533 :param str cert: Certificate (encoded).
1534- :param typ: `OpenSSL.crypto.FILETYPE_PEM` or `OpenSSL.crypto.FILETYPE_ASN1`
1535+ :param typ: `crypto.FILETYPE_PEM` or `crypto.FILETYPE_ASN1`
1536
1537 :returns: A list of Subject Alternative Names.
1538 :rtype: list
1539
1540 """
1541 return _get_sans_from_cert_or_req(
1542- cert, OpenSSL.crypto.load_certificate, typ)
1543+ cert, crypto.load_certificate, typ)
1544
1545
1546 def _get_names_from_cert_or_req(cert_or_req, load_func, typ):
1547@@ -343,24 +367,24 @@ def _get_names_from_loaded_cert_or_req(loaded_cert_or_req):
1548 return acme_crypto_util._pyopenssl_cert_or_req_all_names(loaded_cert_or_req)
1549
1550
1551-def get_names_from_cert(csr, typ=OpenSSL.crypto.FILETYPE_PEM):
1552+def get_names_from_cert(csr, typ=crypto.FILETYPE_PEM):
1553 """Get a list of domains from a cert, including the CN if it is set.
1554
1555 :param str cert: Certificate (encoded).
1556- :param typ: `OpenSSL.crypto.FILETYPE_PEM` or `OpenSSL.crypto.FILETYPE_ASN1`
1557+ :param typ: `crypto.FILETYPE_PEM` or `crypto.FILETYPE_ASN1`
1558
1559 :returns: A list of domain names.
1560 :rtype: list
1561
1562 """
1563 return _get_names_from_cert_or_req(
1564- csr, OpenSSL.crypto.load_certificate, typ)
1565+ csr, crypto.load_certificate, typ)
1566
1567
1568-def dump_pyopenssl_chain(chain, filetype=OpenSSL.crypto.FILETYPE_PEM):
1569+def dump_pyopenssl_chain(chain, filetype=crypto.FILETYPE_PEM):
1570 """Dump certificate chain into a bundle.
1571
1572- :param list chain: List of `OpenSSL.crypto.X509` (or wrapped in
1573+ :param list chain: List of `crypto.X509` (or wrapped in
1574 :class:`josepy.util.ComparableX509`).
1575
1576 """
1577@@ -378,7 +402,7 @@ def notBefore(cert_path):
1578 :rtype: :class:`datetime.datetime`
1579
1580 """
1581- return _notAfterBefore(cert_path, OpenSSL.crypto.X509.get_notBefore)
1582+ return _notAfterBefore(cert_path, crypto.X509.get_notBefore)
1583
1584
1585 def notAfter(cert_path):
1586@@ -390,15 +414,15 @@ def notAfter(cert_path):
1587 :rtype: :class:`datetime.datetime`
1588
1589 """
1590- return _notAfterBefore(cert_path, OpenSSL.crypto.X509.get_notAfter)
1591+ return _notAfterBefore(cert_path, crypto.X509.get_notAfter)
1592
1593
1594 def _notAfterBefore(cert_path, method):
1595 """Internal helper function for finding notbefore/notafter.
1596
1597 :param str cert_path: path to a cert in PEM format
1598- :param function method: one of ``OpenSSL.crypto.X509.get_notBefore``
1599- or ``OpenSSL.crypto.X509.get_notAfter``
1600+ :param function method: one of ``crypto.X509.get_notBefore``
1601+ or ``crypto.X509.get_notAfter``
1602
1603 :returns: the notBefore or notAfter value from the cert at cert_path
1604 :rtype: :class:`datetime.datetime`
1605@@ -406,7 +430,7 @@ def _notAfterBefore(cert_path, method):
1606 """
1607 # pylint: disable=redefined-outer-name
1608 with open(cert_path) as f:
1609- x509 = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM,
1610+ x509 = crypto.load_certificate(crypto.FILETYPE_PEM,
1611 f.read())
1612 # pyopenssl always returns bytes
1613 timestamp = method(x509)
1614@@ -443,7 +467,7 @@ def cert_and_chain_from_fullchain(fullchain_pem):
1615 :rtype: tuple
1616
1617 """
1618- cert = OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM,
1619- OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, fullchain_pem)).decode()
1620+ cert = crypto.dump_certificate(crypto.FILETYPE_PEM,
1621+ crypto.load_certificate(crypto.FILETYPE_PEM, fullchain_pem)).decode()
1622 chain = fullchain_pem[len(cert):].lstrip()
1623 return (cert, chain)
1624diff --git a/certbot/display/ops.py b/certbot/display/ops.py
1625index 57d2362..1e15a84 100644
1626--- a/certbot/display/ops.py
1627+++ b/certbot/display/ops.py
1628@@ -86,13 +86,31 @@ def choose_account(accounts):
1629 else:
1630 return None
1631
1632+def choose_values(values, question=None):
1633+ """Display screen to let user pick one or multiple values from the provided
1634+ list.
1635
1636-def choose_names(installer):
1637+ :param list values: Values to select from
1638+
1639+ :returns: List of selected values
1640+ :rtype: list
1641+ """
1642+ code, items = z_util(interfaces.IDisplay).checklist(
1643+ question, tags=values, force_interactive=True)
1644+ if code == display_util.OK and items:
1645+ return items
1646+ else:
1647+ return []
1648+
1649+def choose_names(installer, question=None):
1650 """Display screen to select domains to validate.
1651
1652 :param installer: An installer object
1653 :type installer: :class:`certbot.interfaces.IInstaller`
1654
1655+ :param `str` question: Overriding dialog question to ask the user if asked
1656+ to choose from domain names.
1657+
1658 :returns: List of selected names
1659 :rtype: `list` of `str`
1660
1661@@ -108,7 +126,7 @@ def choose_names(installer):
1662 return _choose_names_manually(
1663 "No names were found in your configuration files. ")
1664
1665- code, names = _filter_names(names)
1666+ code, names = _filter_names(names, question)
1667 if code == display_util.OK and names:
1668 return names
1669 else:
1670@@ -142,7 +160,7 @@ def _sort_names(FQDNs):
1671 return sorted(FQDNs, key=lambda fqdn: fqdn.split('.')[::-1][1:])
1672
1673
1674-def _filter_names(names):
1675+def _filter_names(names, override_question=None):
1676 """Determine which names the user would like to select from a list.
1677
1678 :param list names: domain names
1679@@ -155,10 +173,12 @@ def _filter_names(names):
1680 """
1681 #Sort by domain first, and then by subdomain
1682 sorted_names = _sort_names(names)
1683-
1684+ if override_question:
1685+ question = override_question
1686+ else:
1687+ question = "Which names would you like to activate HTTPS for?"
1688 code, names = z_util(interfaces.IDisplay).checklist(
1689- "Which names would you like to activate HTTPS for?",
1690- tags=sorted_names, cli_flag="--domains", force_interactive=True)
1691+ question, tags=sorted_names, cli_flag="--domains", force_interactive=True)
1692 return code, [str(s) for s in names]
1693
1694
1695diff --git a/certbot/display/util.py b/certbot/display/util.py
1696index 5e97bca..e157a11 100644
1697--- a/certbot/display/util.py
1698+++ b/certbot/display/util.py
1699@@ -29,6 +29,10 @@ HELP = "help"
1700 ESC = "esc"
1701 """Display exit code when the user hits Escape (UNUSED)"""
1702
1703+# Display constants
1704+SIDE_FRAME = ("- " * 39) + "-"
1705+"""Display boundary (alternates spaces, so when copy-pasted, markdown doesn't interpret
1706+it as a heading)"""
1707
1708 def _wrap_lines(msg):
1709 """Format lines nicely to 80 chars.
1710@@ -111,12 +115,11 @@ class FileDisplay(object):
1711 because it won't cause any workflow regressions
1712
1713 """
1714- side_frame = "-" * 79
1715 if wrap:
1716 message = _wrap_lines(message)
1717 self.outfile.write(
1718 "{line}{frame}{line}{msg}{line}{frame}{line}".format(
1719- line=os.linesep, frame=side_frame, msg=message))
1720+ line=os.linesep, frame=SIDE_FRAME, msg=message))
1721 self.outfile.flush()
1722 if pause:
1723 if self._can_interact(force_interactive):
1724@@ -208,12 +211,10 @@ class FileDisplay(object):
1725 if self._return_default(message, default, cli_flag, force_interactive):
1726 return default
1727
1728- side_frame = ("-" * 79) + os.linesep
1729-
1730 message = _wrap_lines(message)
1731
1732 self.outfile.write("{0}{frame}{msg}{0}{frame}".format(
1733- os.linesep, frame=side_frame, msg=message))
1734+ os.linesep, frame=SIDE_FRAME + os.linesep, msg=message))
1735 self.outfile.flush()
1736
1737 while True:
1738@@ -386,8 +387,7 @@ class FileDisplay(object):
1739 # Write out the message to the user
1740 self.outfile.write(
1741 "{new}{msg}{new}".format(new=os.linesep, msg=message))
1742- side_frame = ("-" * 79) + os.linesep
1743- self.outfile.write(side_frame)
1744+ self.outfile.write(SIDE_FRAME + os.linesep)
1745
1746 # Write out the menu choices
1747 for i, desc in enumerate(choices, 1):
1748@@ -397,7 +397,7 @@ class FileDisplay(object):
1749 # Keep this outside of the textwrap
1750 self.outfile.write(os.linesep)
1751
1752- self.outfile.write(side_frame)
1753+ self.outfile.write(SIDE_FRAME + os.linesep)
1754 self.outfile.flush()
1755
1756 def _get_valid_int_ans(self, max_):
1757@@ -482,12 +482,11 @@ class NoninteractiveDisplay(object):
1758 :param bool wrap: Whether or not the application should wrap text
1759
1760 """
1761- side_frame = "-" * 79
1762 if wrap:
1763 message = _wrap_lines(message)
1764 self.outfile.write(
1765 "{line}{frame}{line}{msg}{line}{frame}{line}".format(
1766- line=os.linesep, frame=side_frame, msg=message))
1767+ line=os.linesep, frame=SIDE_FRAME, msg=message))
1768 self.outfile.flush()
1769
1770 def menu(self, message, choices, ok_label=None, cancel_label=None,
1771diff --git a/certbot/eff.py b/certbot/eff.py
1772index 746261f..388ae98 100644
1773--- a/certbot/eff.py
1774+++ b/certbot/eff.py
1775@@ -41,8 +41,8 @@ def _want_subscription():
1776 'Would you be willing to share your email address with the '
1777 "Electronic Frontier Foundation, a founding partner of the Let's "
1778 'Encrypt project and the non-profit organization that develops '
1779- "Certbot? We'd like to send you email about EFF and our work to "
1780- 'encrypt the web, protect its users and defend digital rights.')
1781+ "Certbot? We'd like to send you email about our work encrypting "
1782+ "the web, EFF news, campaigns, and ways to support digital freedom. ")
1783 display = zope.component.getUtility(interfaces.IDisplay)
1784 return display.yesno(prompt, default=False)
1785
1786@@ -71,11 +71,14 @@ def _check_response(response):
1787
1788 """
1789 logger.debug('Received response:\n%s', response.content)
1790- if response.ok:
1791- if not response.json()['status']:
1792+ try:
1793+ response.raise_for_status()
1794+ if response.json()['status'] == False:
1795 _report_failure('your e-mail address appears to be invalid')
1796- else:
1797+ except requests.exceptions.HTTPError:
1798 _report_failure()
1799+ except (ValueError, KeyError):
1800+ _report_failure('there was a problem with the server response')
1801
1802
1803 def _report_failure(reason=None):
1804diff --git a/certbot/error_handler.py b/certbot/error_handler.py
1805index e273771..5e72f81 100644
1806--- a/certbot/error_handler.py
1807+++ b/certbot/error_handler.py
1808@@ -5,6 +5,10 @@ import os
1809 import signal
1810 import traceback
1811
1812+# pylint: disable=unused-import, no-name-in-module
1813+from acme.magic_typing import Any, Callable, Dict, List, Union
1814+# pylint: enable=unused-import, no-name-in-module
1815+
1816 from certbot import errors
1817
1818 logger = logging.getLogger(__name__)
1819@@ -56,9 +60,9 @@ class ErrorHandler(object):
1820 def __init__(self, func=None, *args, **kwargs):
1821 self.call_on_regular_exit = False
1822 self.body_executed = False
1823- self.funcs = []
1824- self.prev_handlers = {}
1825- self.received_signals = []
1826+ self.funcs = [] # type: List[Callable[[], Any]]
1827+ self.prev_handlers = {} # type: Dict[int, Union[int, None, Callable]]
1828+ self.received_signals = [] # type: List[int]
1829 if func is not None:
1830 self.register(func, *args, **kwargs)
1831
1832@@ -88,6 +92,7 @@ class ErrorHandler(object):
1833 return retval
1834
1835 def register(self, func, *args, **kwargs):
1836+ # type: (Callable, *Any, **Any) -> None
1837 """Sets func to be run with the given arguments during cleanup.
1838
1839 :param function func: function to be called in case of an error
1840@@ -101,9 +106,8 @@ class ErrorHandler(object):
1841 while self.funcs:
1842 try:
1843 self.funcs[-1]()
1844- except Exception as error: # pylint: disable=broad-except
1845- logger.error("Encountered exception during recovery")
1846- logger.exception(error)
1847+ except Exception: # pylint: disable=broad-except
1848+ logger.error("Encountered exception during recovery: ", exc_info=True)
1849 self.funcs.pop()
1850
1851 def _set_signal_handlers(self):
1852diff --git a/certbot/errors.py b/certbot/errors.py
1853index e9c4a08..48aebc2 100644
1854--- a/certbot/errors.py
1855+++ b/certbot/errors.py
1856@@ -87,6 +87,10 @@ class NotSupportedError(PluginError):
1857 """Certbot Plugin function not supported error."""
1858
1859
1860+class PluginStorageError(PluginError):
1861+ """Certbot Plugin Storage error."""
1862+
1863+
1864 class StandaloneBindError(Error):
1865 """Standalone plugin bind error."""
1866
1867diff --git a/certbot/hooks.py b/certbot/hooks.py
1868index b5c9046..d5239a4 100644
1869--- a/certbot/hooks.py
1870+++ b/certbot/hooks.py
1871@@ -6,6 +6,7 @@ import os
1872
1873 from subprocess import Popen, PIPE
1874
1875+from acme.magic_typing import Set, List # pylint: disable=unused-import, no-name-in-module
1876 from certbot import errors
1877 from certbot import util
1878
1879@@ -76,7 +77,8 @@ def pre_hook(config):
1880 if cmd:
1881 _run_pre_hook_if_necessary(cmd)
1882
1883-pre_hook.already = set() # type: ignore
1884+
1885+executed_pre_hooks = set() # type: Set[str]
1886
1887
1888 def _run_pre_hook_if_necessary(command):
1889@@ -88,12 +90,12 @@ def _run_pre_hook_if_necessary(command):
1890 :param str command: pre-hook to be run
1891
1892 """
1893- if command in pre_hook.already:
1894+ if command in executed_pre_hooks:
1895 logger.info("Pre-hook command already run, skipping: %s", command)
1896 else:
1897 logger.info("Running pre-hook command: %s", command)
1898 _run_hook(command)
1899- pre_hook.already.add(command)
1900+ executed_pre_hooks.add(command)
1901
1902
1903 def post_hook(config):
1904@@ -127,7 +129,8 @@ def post_hook(config):
1905 logger.info("Running post-hook command: %s", cmd)
1906 _run_hook(cmd)
1907
1908-post_hook.eventually = [] # type: ignore
1909+
1910+post_hooks = [] # type: List[str]
1911
1912
1913 def _run_eventually(command):
1914@@ -139,13 +142,13 @@ def _run_eventually(command):
1915 :param str command: post-hook to register to be run
1916
1917 """
1918- if command not in post_hook.eventually:
1919- post_hook.eventually.append(command)
1920+ if command not in post_hooks:
1921+ post_hooks.append(command)
1922
1923
1924 def run_saved_post_hooks():
1925 """Run any post hooks that were saved up in the course of the 'renew' verb"""
1926- for cmd in post_hook.eventually:
1927+ for cmd in post_hooks:
1928 logger.info("Running post-hook command: %s", cmd)
1929 _run_hook(cmd)
1930
1931diff --git a/certbot/interfaces.py b/certbot/interfaces.py
1932index 501a5c5..2e837d1 100644
1933--- a/certbot/interfaces.py
1934+++ b/certbot/interfaces.py
1935@@ -1,16 +1,16 @@
1936 """Certbot client interfaces."""
1937 import abc
1938+import six
1939 import zope.interface
1940
1941 # pylint: disable=no-self-argument,no-method-argument,no-init,inherit-non-class
1942 # pylint: disable=too-few-public-methods
1943
1944
1945+@six.add_metaclass(abc.ABCMeta)
1946 class AccountStorage(object):
1947 """Accounts storage interface."""
1948
1949- __metaclass__ = abc.ABCMeta
1950-
1951 @abc.abstractmethod
1952 def find_all(self): # pragma: no cover
1953 """Find all accounts.
1954@@ -201,7 +201,9 @@ class IConfig(zope.interface.Interface):
1955 """
1956 server = zope.interface.Attribute("ACME Directory Resource URI.")
1957 email = zope.interface.Attribute(
1958- "Email used for registration and recovery contact. (default: Ask)")
1959+ "Email used for registration and recovery contact. Use comma to "
1960+ "register multiple emails, ex: u1@example.com,u2@example.com. "
1961+ "(default: Ask).")
1962 rsa_key_size = zope.interface.Attribute("Size of the RSA key.")
1963 must_staple = zope.interface.Attribute(
1964 "Adds the OCSP Must Staple extension to the certificate. "
1965@@ -256,6 +258,10 @@ class IConfig(zope.interface.Interface):
1966 "user; only needed if your config is somewhere unsafe like /tmp/."
1967 "This is a boolean")
1968
1969+ disable_renew_updates = zope.interface.Attribute(
1970+ "If updates provided by installer enhancements when Certbot is being run"
1971+ " with \"renew\" verb should be disabled.")
1972+
1973 class IInstaller(IPlugin):
1974 """Generic Certbot Installer Interface.
1975
1976@@ -591,3 +597,75 @@ class IReporter(zope.interface.Interface):
1977
1978 def print_messages(self):
1979 """Prints messages to the user and clears the message queue."""
1980+
1981+
1982+# Updater interfaces
1983+#
1984+# When "certbot renew" is run, Certbot will iterate over each lineage and check
1985+# if the selected installer for that lineage is a subclass of each updater
1986+# class. If it is and the update of that type is configured to be run for that
1987+# lineage, the relevant update function will be called for it. These functions
1988+# are never called for other subcommands, so if an installer wants to perform
1989+# an update during the run or install subcommand, it should do so when
1990+# :func:`IInstaller.deploy_cert` is called.
1991+
1992+@six.add_metaclass(abc.ABCMeta)
1993+class GenericUpdater(object):
1994+ """Interface for update types not currently specified by Certbot.
1995+
1996+ This class allows plugins to perform types of updates that Certbot hasn't
1997+ defined (yet).
1998+
1999+ To make use of this interface, the installer should implement the interface
2000+ methods, and interfaces.GenericUpdater.register(InstallerClass) should
2001+ be called from the installer code.
2002+
2003+ The plugins implementing this enhancement are responsible of handling
2004+ the saving of configuration checkpoints as well as other calls to
2005+ interface methods of `interfaces.IInstaller` such as prepare() and restart()
2006+ """
2007+
2008+ @abc.abstractmethod
2009+ def generic_updates(self, lineage, *args, **kwargs):
2010+ """Perform any update types defined by the installer.
2011+
2012+ If an installer is a subclass of the class containing this method, this
2013+ function will always be called when "certbot renew" is run. If the
2014+ update defined by the installer should be run conditionally, the
2015+ installer needs to handle checking the conditions itself.
2016+
2017+ This method is called once for each lineage.
2018+
2019+ :param lineage: Certificate lineage object
2020+ :type lineage: storage.RenewableCert
2021+
2022+ """
2023+
2024+
2025+@six.add_metaclass(abc.ABCMeta)
2026+class RenewDeployer(object):
2027+ """Interface for update types run when a lineage is renewed
2028+
2029+ This class allows plugins to perform types of updates that need to run at
2030+ lineage renewal that Certbot hasn't defined (yet).
2031+
2032+ To make use of this interface, the installer should implement the interface
2033+ methods, and interfaces.RenewDeployer.register(InstallerClass) should
2034+ be called from the installer code.
2035+ """
2036+
2037+ @abc.abstractmethod
2038+ def renew_deploy(self, lineage, *args, **kwargs):
2039+ """Perform updates defined by installer when a certificate has been renewed
2040+
2041+ If an installer is a subclass of the class containing this method, this
2042+ function will always be called when a certficate has been renewed by
2043+ running "certbot renew". For example if a plugin needs to copy a
2044+ certificate over, or change configuration based on the new certificate.
2045+
2046+ This method is called once for each lineage renewed
2047+
2048+ :param lineage: Certificate lineage object
2049+ :type lineage: storage.RenewableCert
2050+
2051+ """
2052diff --git a/certbot/log.py b/certbot/log.py
2053index face93c..89626af 100644
2054--- a/certbot/log.py
2055+++ b/certbot/log.py
2056@@ -191,9 +191,8 @@ class MemoryHandler(logging.handlers.MemoryHandler):
2057 only happens when flush(force=True) is called.
2058
2059 """
2060- def __init__(self, target=None):
2061+ def __init__(self, target=None, capacity=10000):
2062 # capacity doesn't matter because should_flush() is overridden
2063- capacity = float('inf')
2064 super(MemoryHandler, self).__init__(capacity, target=target)
2065
2066 def close(self):
2067diff --git a/certbot/main.py b/certbot/main.py
2068index 7be852e..2cba874 100644
2069--- a/certbot/main.py
2070+++ b/certbot/main.py
2071@@ -11,6 +11,7 @@ import josepy as jose
2072 import zope.component
2073
2074 from acme import errors as acme_errors
2075+from acme.magic_typing import Union # pylint: disable=unused-import, no-name-in-module
2076
2077 import certbot
2078
2079@@ -29,12 +30,13 @@ from certbot import log
2080 from certbot import renewal
2081 from certbot import reporter
2082 from certbot import storage
2083+from certbot import updater
2084 from certbot import util
2085
2086 from certbot.display import util as display_util, ops as display_ops
2087 from certbot.plugins import disco as plugins_disco
2088 from certbot.plugins import selection as plug_sel
2089-
2090+from certbot.plugins import enhancements
2091
2092 USER_CANCELLED = ("User chose to cancel the operation and may "
2093 "reinvoke the client.")
2094@@ -323,7 +325,7 @@ def _find_lineage_for_domains_and_certname(config, domains, certname):
2095 return "newcert", None
2096 else:
2097 raise errors.ConfigurationError("No certificate with name {0} found. "
2098- "Use -d to specify domains, or run certbot --certificates to see "
2099+ "Use -d to specify domains, or run certbot certificates to see "
2100 "possible certificate names.".format(certname))
2101
2102 def _get_added_removed(after, before):
2103@@ -339,7 +341,10 @@ def _get_added_removed(after, before):
2104 def _format_list(character, strings):
2105 """Format list with given character
2106 """
2107- formatted = "{br}{ch} " + "{br}{ch} ".join(strings)
2108+ if len(strings) == 0:
2109+ formatted = "{br}(None)"
2110+ else:
2111+ formatted = "{br}{ch} " + "{br}{ch} ".join(strings)
2112 return formatted.format(
2113 ch=character,
2114 br=os.linesep
2115@@ -382,7 +387,7 @@ def _ask_user_to_confirm_new_names(config, new_domains, certname, old_domains):
2116 if not obj.yesno(msg, "Update cert", "Cancel", default=True):
2117 raise errors.ConfigurationError("Specified mismatched cert name and domains.")
2118
2119-def _find_domains_or_certname(config, installer):
2120+def _find_domains_or_certname(config, installer, question=None):
2121 """Retrieve domains and certname from config or user input.
2122
2123 :param config: Configuration object
2124@@ -391,6 +396,8 @@ def _find_domains_or_certname(config, installer):
2125 :param installer: Installer object
2126 :type installer: interfaces.IInstaller
2127
2128+ :param `str` question: Overriding dialog question to ask the user if asked
2129+ to choose from domain names.
2130
2131 :returns: Two-part tuple of domains and certname
2132 :rtype: `tuple` of list of `str` and `str`
2133@@ -411,7 +418,7 @@ def _find_domains_or_certname(config, installer):
2134 # that certname might not have existed, or there was a problem.
2135 # try to get domains from the user.
2136 if not domains:
2137- domains = display_ops.choose_names(installer)
2138+ domains = display_ops.choose_names(installer, question)
2139
2140 if not domains and not certname:
2141 raise errors.Error("Please specify --domains, or --installer that "
2142@@ -466,8 +473,7 @@ def _report_new_cert(config, cert_path, fullchain_path, key_path=None):
2143 def _determine_account(config):
2144 """Determine which account to use.
2145
2146- In order to make the renewer (configuration de/serialization) happy,
2147- if ``config.account`` is ``None``, it will be updated based on the
2148+ If ``config.account`` is ``None``, it will be updated based on the
2149 user input. Same for ``config.email``.
2150
2151 :param config: Configuration object
2152@@ -480,6 +486,21 @@ def _determine_account(config):
2153 :raises errors.Error: If unable to register an account with ACME server
2154
2155 """
2156+ def _tos_cb(terms_of_service):
2157+ if config.tos:
2158+ return True
2159+ msg = ("Please read the Terms of Service at {0}. You "
2160+ "must agree in order to register with the ACME "
2161+ "server at {1}".format(
2162+ terms_of_service, config.server))
2163+ obj = zope.component.getUtility(interfaces.IDisplay)
2164+ result = obj.yesno(msg, "Agree", "Cancel",
2165+ cli_flag="--agree-tos", force_interactive=True)
2166+ if not result:
2167+ raise errors.Error(
2168+ "Registration cannot proceed without accepting "
2169+ "Terms of Service.")
2170+
2171 account_storage = account.AccountFileStorage(config)
2172 acme = None
2173
2174@@ -494,28 +515,13 @@ def _determine_account(config):
2175 else: # no account registered yet
2176 if config.email is None and not config.register_unsafely_without_email:
2177 config.email = display_ops.get_email()
2178-
2179- def _tos_cb(terms_of_service):
2180- if config.tos:
2181- return True
2182- msg = ("Please read the Terms of Service at {0}. You "
2183- "must agree in order to register with the ACME "
2184- "server at {1}".format(
2185- terms_of_service, config.server))
2186- obj = zope.component.getUtility(interfaces.IDisplay)
2187- result = obj.yesno(msg, "Agree", "Cancel",
2188- cli_flag="--agree-tos", force_interactive=True)
2189- if not result:
2190- raise errors.Error(
2191- "Registration cannot proceed without accepting "
2192- "Terms of Service.")
2193 try:
2194 acc, acme = client.register(
2195 config, account_storage, tos_cb=_tos_cb)
2196 except errors.MissingCommandlineFlag:
2197 raise
2198- except errors.Error as error:
2199- logger.debug(error, exc_info=True)
2200+ except errors.Error:
2201+ logger.debug("", exc_info=True)
2202 raise errors.Error(
2203 "Unable to register an account with ACME server")
2204
2205@@ -728,8 +734,14 @@ def register(config, unused_plugins):
2206 acc, acme = _determine_account(config)
2207 cb_client = client.Client(config, acc, None, None, acme=acme)
2208 # We rely on an exception to interrupt this process if it didn't work.
2209+ acc_contacts = ['mailto:' + email for email in config.email.split(',')]
2210+ prev_regr_uri = acc.regr.uri
2211 acc.regr = cb_client.acme.update_registration(acc.regr.update(
2212- body=acc.regr.body.update(contact=('mailto:' + config.email,))))
2213+ body=acc.regr.body.update(contact=acc_contacts)))
2214+ # A v1 account being used as a v2 account will result in changing the uri to
2215+ # the v2 uri. Since it's the same object on disk, put it back to the v1 uri
2216+ # so that we can also continue to use the account object with acmev1.
2217+ acc.regr = acc.regr.update(uri=prev_regr_uri)
2218 account_storage.save_regr(acc, cb_client.acme)
2219 eff.handle_subscription(config)
2220 add_msg("Your e-mail address was updated to {0}.".format(config.email))
2221@@ -743,8 +755,8 @@ def _install_cert(config, le_client, domains, lineage=None):
2222 :param le_client: Client object
2223 :type le_client: client.Client
2224
2225- :param plugins: List of domains
2226- :type plugins: `list` of `str`
2227+ :param domains: List of domains
2228+ :type domains: `list` of `str`
2229
2230 :param lineage: Certificate lineage object. Defaults to `None`
2231 :type lineage: storage.RenewableCert
2232@@ -782,11 +794,26 @@ def install(config, plugins):
2233 except errors.PluginSelectionError as e:
2234 return str(e)
2235
2236+ custom_cert = (config.key_path and config.cert_path)
2237+ if not config.certname and not custom_cert:
2238+ certname_question = "Which certificate would you like to install?"
2239+ config.certname = cert_manager.get_certnames(
2240+ config, "install", allow_multiple=False,
2241+ custom_prompt=certname_question)[0]
2242+
2243+ if not enhancements.are_supported(config, installer):
2244+ raise errors.NotSupportedError("One ore more of the requested enhancements "
2245+ "are not supported by the selected installer")
2246 # If cert-path is defined, populate missing (ie. not overridden) values.
2247 # Unfortunately this can't be done in argument parser, as certificate
2248 # manager needs the access to renewal directory paths
2249 if config.certname:
2250 config = _populate_from_certname(config)
2251+ elif enhancements.are_requested(config):
2252+ # Preflight config check
2253+ raise errors.ConfigurationError("One or more of the requested enhancements "
2254+ "require --cert-name to be provided")
2255+
2256 if config.key_path and config.cert_path:
2257 _check_certificate_and_key(config)
2258 domains, _ = _find_domains_or_certname(config, installer)
2259@@ -797,6 +824,11 @@ def install(config, plugins):
2260 "If your certificate is managed by Certbot, please use --cert-name "
2261 "to define which certificate you would like to install.")
2262
2263+ if enhancements.are_requested(config):
2264+ # In the case where we don't have certname, we have errored out already
2265+ lineage = cert_manager.lineage_for_certname(config, config.certname)
2266+ enhancements.enable(lineage, domains, installer, config)
2267+
2268 def _populate_from_certname(config):
2269 """Helper function for install to populate missing config values from lineage
2270 defined by --cert-name."""
2271@@ -859,6 +891,62 @@ def plugins_cmd(config, plugins):
2272 logger.debug("Prepared plugins: %s", available)
2273 notify(str(available))
2274
2275+def enhance(config, plugins):
2276+ """Add security enhancements to existing configuration
2277+
2278+ :param config: Configuration object
2279+ :type config: interfaces.IConfig
2280+
2281+ :param plugins: List of plugins
2282+ :type plugins: `list` of `str`
2283+
2284+ :returns: `None`
2285+ :rtype: None
2286+
2287+ """
2288+ supported_enhancements = ["hsts", "redirect", "uir", "staple"]
2289+ # Check that at least one enhancement was requested on command line
2290+ oldstyle_enh = any([getattr(config, enh) for enh in supported_enhancements])
2291+ if not enhancements.are_requested(config) and not oldstyle_enh:
2292+ msg = ("Please specify one or more enhancement types to configure. To list "
2293+ "the available enhancement types, run:\n\n%s --help enhance\n")
2294+ logger.warning(msg, sys.argv[0])
2295+ raise errors.MisconfigurationError("No enhancements requested, exiting.")
2296+
2297+ try:
2298+ installer, _ = plug_sel.choose_configurator_plugins(config, plugins, "enhance")
2299+ except errors.PluginSelectionError as e:
2300+ return str(e)
2301+
2302+ if not enhancements.are_supported(config, installer):
2303+ raise errors.NotSupportedError("One ore more of the requested enhancements "
2304+ "are not supported by the selected installer")
2305+
2306+ certname_question = ("Which certificate would you like to use to enhance "
2307+ "your configuration?")
2308+ config.certname = cert_manager.get_certnames(
2309+ config, "enhance", allow_multiple=False,
2310+ custom_prompt=certname_question)[0]
2311+ cert_domains = cert_manager.domains_for_certname(config, config.certname)
2312+ if config.noninteractive_mode:
2313+ domains = cert_domains
2314+ else:
2315+ domain_question = ("Which domain names would you like to enable the "
2316+ "selected enhancements for?")
2317+ domains = display_ops.choose_values(cert_domains, domain_question)
2318+ if not domains:
2319+ raise errors.Error("User cancelled the domain selection. No domains "
2320+ "defined, exiting.")
2321+
2322+ lineage = cert_manager.lineage_for_certname(config, config.certname)
2323+ if not config.chain_path:
2324+ config.chain_path = lineage.chain_path
2325+ if oldstyle_enh:
2326+ le_client = _init_le_client(config, authenticator=None, installer=installer)
2327+ le_client.enhance_config(domains, config.chain_path, ask_redirect=False)
2328+ if enhancements.are_requested(config):
2329+ enhancements.enable(lineage, domains, installer, config)
2330+
2331
2332 def rollback(config, plugins):
2333 """Rollback server configuration changes made during install.
2334@@ -976,7 +1064,7 @@ def revoke(config, unused_plugins): # TODO: coop with renewal config
2335
2336 """
2337 # For user-agent construction
2338- config.installer = config.authenticator = "None"
2339+ config.installer = config.authenticator = None
2340 if config.key_path is not None: # revocation by cert key
2341 logger.debug("Revoking %s using cert key %s",
2342 config.cert_path[0], config.key_path[0])
2343@@ -1019,6 +1107,11 @@ def run(config, plugins): # pylint: disable=too-many-branches,too-many-locals
2344 except errors.PluginSelectionError as e:
2345 return str(e)
2346
2347+ # Preflight check for enhancement support by the selected installer
2348+ if not enhancements.are_supported(config, installer):
2349+ raise errors.NotSupportedError("One ore more of the requested enhancements "
2350+ "are not supported by the selected installer")
2351+
2352 # TODO: Handle errors from _init_le_client?
2353 le_client = _init_le_client(config, authenticator, installer)
2354
2355@@ -1037,6 +1130,9 @@ def run(config, plugins): # pylint: disable=too-many-branches,too-many-locals
2356
2357 _install_cert(config, le_client, domains, new_lineage)
2358
2359+ if enhancements.are_requested(config) and new_lineage:
2360+ enhancements.enable(new_lineage, domains, installer, config)
2361+
2362 if lineage is None or not should_get_cert:
2363 display_ops.success_installation(domains)
2364 else:
2365@@ -1096,10 +1192,9 @@ def renew_cert(config, plugins, lineage):
2366 except errors.PluginSelectionError as e:
2367 logger.info("Could not choose appropriate plugin: %s", e)
2368 raise
2369-
2370 le_client = _init_le_client(config, auth, installer)
2371
2372- _get_and_save_cert(le_client, config, lineage=lineage)
2373+ renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
2374
2375 notify = zope.component.getUtility(interfaces.IDisplay).notification
2376 if installer is None:
2377@@ -1109,6 +1204,8 @@ def renew_cert(config, plugins, lineage):
2378 # In case of a renewal, reload server to pick up new certificate.
2379 # In principle we could have a configuration option to inhibit this
2380 # from happening.
2381+ # Run deployer
2382+ updater.run_renewal_deployer(config, renewed_lineage, installer)
2383 installer.restart()
2384 notify("new certificate deployed with reload of {0} server; fullchain is {1}".format(
2385 config.installer, lineage.fullchain), pause=False)
2386@@ -1217,7 +1314,8 @@ def set_displayer(config):
2387 """
2388 if config.quiet:
2389 config.noninteractive_mode = True
2390- displayer = display_util.NoninteractiveDisplay(open(os.devnull, "w"))
2391+ displayer = display_util.NoninteractiveDisplay(open(os.devnull, "w")) \
2392+ # type: Union[None, display_util.NoninteractiveDisplay, display_util.FileDisplay]
2393 elif config.noninteractive_mode:
2394 displayer = display_util.NoninteractiveDisplay(sys.stdout)
2395 else:
2396diff --git a/certbot/plugins/common.py b/certbot/plugins/common.py
2397index c281534..ee1af49 100644
2398--- a/certbot/plugins/common.py
2399+++ b/certbot/plugins/common.py
2400@@ -11,6 +11,8 @@ import zope.interface
2401
2402 from josepy import util as jose_util
2403
2404+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
2405+from certbot import achallenges # pylint: disable=unused-import
2406 from certbot import constants
2407 from certbot import crypto_util
2408 from certbot import errors
2409@@ -18,6 +20,8 @@ from certbot import interfaces
2410 from certbot import reverter
2411 from certbot import util
2412
2413+from certbot.plugins.storage import PluginStorage
2414+
2415 logger = logging.getLogger(__name__)
2416
2417
2418@@ -99,7 +103,6 @@ class Plugin(object):
2419 def conf(self, var):
2420 """Find a configuration value for variable ``var``."""
2421 return getattr(self.config, self.dest(var))
2422-# other
2423
2424
2425 class Installer(Plugin):
2426@@ -110,6 +113,7 @@ class Installer(Plugin):
2427 """
2428 def __init__(self, *args, **kwargs):
2429 super(Installer, self).__init__(*args, **kwargs)
2430+ self.storage = PluginStorage(self.config, self.name)
2431 self.reverter = reverter.Reverter(self.config)
2432
2433 def add_to_checkpoint(self, save_files, save_notes, temporary=False):
2434@@ -329,8 +333,8 @@ class ChallengePerformer(object):
2435
2436 def __init__(self, configurator):
2437 self.configurator = configurator
2438- self.achalls = []
2439- self.indices = []
2440+ self.achalls = [] # type: List[achallenges.KeyAuthorizationAnnotatedChallenge]
2441+ self.indices = [] # type: List[int]
2442
2443 def add_chall(self, achall, idx=None):
2444 """Store challenge to be performed when perform() is called.
2445diff --git a/certbot/plugins/disco.py b/certbot/plugins/disco.py
2446index 062c116..7be320e 100644
2447--- a/certbot/plugins/disco.py
2448+++ b/certbot/plugins/disco.py
2449@@ -10,6 +10,7 @@ from collections import OrderedDict
2450 import zope.interface
2451 import zope.interface.verify
2452
2453+from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
2454 from certbot import constants
2455 from certbot import errors
2456 from certbot import interfaces
2457@@ -29,12 +30,17 @@ class PluginEntryPoint(object):
2458 "certbot-dns-digitalocean",
2459 "certbot-dns-dnsimple",
2460 "certbot-dns-dnsmadeeasy",
2461+ "certbot-dns-gehirn",
2462 "certbot-dns-google",
2463+ "certbot-dns-linode",
2464 "certbot-dns-luadns",
2465 "certbot-dns-nsone",
2466+ "certbot-dns-ovh",
2467 "certbot-dns-rfc2136",
2468 "certbot-dns-route53",
2469+ "certbot-dns-sakuracloud",
2470 "certbot-nginx",
2471+ "certbot-postfix",
2472 ]
2473 """Distributions for which prefix will be omitted."""
2474
2475@@ -189,7 +195,7 @@ class PluginsRegistry(collections.Mapping):
2476 @classmethod
2477 def find_all(cls):
2478 """Find plugins using setuptools entry points."""
2479- plugins = {}
2480+ plugins = {} # type: Dict[str, PluginEntryPoint]
2481 # pylint: disable=not-callable
2482 entry_points = itertools.chain(
2483 pkg_resources.iter_entry_points(
2484diff --git a/certbot/plugins/disco_test.py b/certbot/plugins/disco_test.py
2485index 220b902..720b90b 100644
2486--- a/certbot/plugins/disco_test.py
2487+++ b/certbot/plugins/disco_test.py
2488@@ -8,6 +8,7 @@ import pkg_resources
2489 import six
2490 import zope.interface
2491
2492+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
2493 from certbot import errors
2494 from certbot import interfaces
2495
2496@@ -250,7 +251,7 @@ class PluginsRegistryTest(unittest.TestCase):
2497 self.plugin_ep.prepare.assert_called_once_with()
2498
2499 def test_prepare_order(self):
2500- order = []
2501+ order = [] # type: List[str]
2502 plugins = dict(
2503 (c, mock.MagicMock(prepare=functools.partial(order.append, c)))
2504 for c in string.ascii_letters)
2505diff --git a/certbot/plugins/enhancements.py b/certbot/plugins/enhancements.py
2506new file mode 100644
2507index 0000000..7ca0966
2508--- /dev/null
2509+++ b/certbot/plugins/enhancements.py
2510@@ -0,0 +1,164 @@
2511+"""New interface style Certbot enhancements"""
2512+import abc
2513+import six
2514+
2515+from certbot import constants
2516+
2517+from acme.magic_typing import Dict, List, Any # pylint: disable=unused-import, no-name-in-module
2518+
2519+def enabled_enhancements(config):
2520+ """
2521+ Generator to yield the enabled new style enhancements.
2522+
2523+ :param config: Configuration.
2524+ :type config: :class:`certbot.interfaces.IConfig`
2525+ """
2526+ for enh in _INDEX:
2527+ if getattr(config, enh["cli_dest"]):
2528+ yield enh
2529+
2530+def are_requested(config):
2531+ """
2532+ Checks if one or more of the requested enhancements are those of the new
2533+ enhancement interfaces.
2534+
2535+ :param config: Configuration.
2536+ :type config: :class:`certbot.interfaces.IConfig`
2537+ """
2538+ return any(enabled_enhancements(config))
2539+
2540+def are_supported(config, installer):
2541+ """
2542+ Checks that all of the requested enhancements are supported by the
2543+ installer.
2544+
2545+ :param config: Configuration.
2546+ :type config: :class:`certbot.interfaces.IConfig`
2547+
2548+ :param installer: Installer object
2549+ :type installer: interfaces.IInstaller
2550+
2551+ :returns: If all the requested enhancements are supported by the installer
2552+ :rtype: bool
2553+ """
2554+ for enh in enabled_enhancements(config):
2555+ if not isinstance(installer, enh["class"]):
2556+ return False
2557+ return True
2558+
2559+def enable(lineage, domains, installer, config):
2560+ """
2561+ Run enable method for each requested enhancement that is supported.
2562+
2563+ :param lineage: Certificate lineage object
2564+ :type lineage: certbot.storage.RenewableCert
2565+
2566+ :param domains: List of domains in certificate to enhance
2567+ :type domains: str
2568+
2569+ :param installer: Installer object
2570+ :type installer: interfaces.IInstaller
2571+
2572+ :param config: Configuration.
2573+ :type config: :class:`certbot.interfaces.IConfig`
2574+ """
2575+ for enh in enabled_enhancements(config):
2576+ getattr(installer, enh["enable_function"])(lineage, domains)
2577+
2578+def populate_cli(add):
2579+ """
2580+ Populates the command line flags for certbot.cli.HelpfulParser
2581+
2582+ :param add: Add function of certbot.cli.HelpfulParser
2583+ :type add: func
2584+ """
2585+ for enh in _INDEX:
2586+ add(enh["cli_groups"], enh["cli_flag"], action=enh["cli_action"],
2587+ dest=enh["cli_dest"], default=enh["cli_flag_default"],
2588+ help=enh["cli_help"])
2589+
2590+
2591+@six.add_metaclass(abc.ABCMeta)
2592+class AutoHSTSEnhancement(object):
2593+ """
2594+ Enhancement interface that installer plugins can implement in order to
2595+ provide functionality that configures the software to have a
2596+ 'Strict-Transport-Security' with initially low max-age value that will
2597+ increase over time.
2598+
2599+ The plugins implementing new style enhancements are responsible of handling
2600+ the saving of configuration checkpoints as well as calling possible restarts
2601+ of managed software themselves. For update_autohsts method, the installer may
2602+ have to call prepare() to finalize the plugin initialization.
2603+
2604+ Methods:
2605+ enable_autohsts is called when the header is initially installed using a
2606+ low max-age value.
2607+
2608+ update_autohsts is called every time when Certbot is run using 'renew'
2609+ verb. The max-age value should be increased over time using this method.
2610+
2611+ deploy_autohsts is called for every lineage that has had its certificate
2612+ renewed. A long HSTS max-age value should be set here, as we should be
2613+ confident that the user is able to automatically renew their certificates.
2614+
2615+
2616+ """
2617+
2618+ @abc.abstractmethod
2619+ def update_autohsts(self, lineage, *args, **kwargs):
2620+ """
2621+ Gets called for each lineage every time Certbot is run with 'renew' verb.
2622+ Implementation of this method should increase the max-age value.
2623+
2624+ :param lineage: Certificate lineage object
2625+ :type lineage: certbot.storage.RenewableCert
2626+
2627+ .. note:: prepare() method inherited from `interfaces.IPlugin` might need
2628+ to be called manually within implementation of this interface method
2629+ to finalize the plugin initialization.
2630+ """
2631+
2632+ @abc.abstractmethod
2633+ def deploy_autohsts(self, lineage, *args, **kwargs):
2634+ """
2635+ Gets called for a lineage when its certificate is successfully renewed.
2636+ Long max-age value should be set in implementation of this method.
2637+
2638+ :param lineage: Certificate lineage object
2639+ :type lineage: certbot.storage.RenewableCert
2640+ """
2641+
2642+ @abc.abstractmethod
2643+ def enable_autohsts(self, lineage, domains, *args, **kwargs):
2644+ """
2645+ Enables the AutoHSTS enhancement, installing
2646+ Strict-Transport-Security header with a low initial value to be increased
2647+ over the subsequent runs of Certbot renew.
2648+
2649+ :param lineage: Certificate lineage object
2650+ :type lineage: certbot.storage.RenewableCert
2651+
2652+ :param domains: List of domains in certificate to enhance
2653+ :type domains: str
2654+ """
2655+
2656+# This is used to configure internal new style enhancements in Certbot. These
2657+# enhancement interfaces need to be defined in this file. Please do not modify
2658+# this list from plugin code.
2659+_INDEX = [
2660+ {
2661+ "name": "AutoHSTS",
2662+ "cli_help": "Gradually increasing max-age value for HTTP Strict Transport "+
2663+ "Security security header",
2664+ "cli_flag": "--auto-hsts",
2665+ "cli_flag_default": constants.CLI_DEFAULTS["auto_hsts"],
2666+ "cli_groups": ["security", "enhance"],
2667+ "cli_dest": "auto_hsts",
2668+ "cli_action": "store_true",
2669+ "class": AutoHSTSEnhancement,
2670+ "updater_function": "update_autohsts",
2671+ "deployer_function": "deploy_autohsts",
2672+ "enable_function": "enable_autohsts"
2673+ }
2674+] # type: List[Dict[str, Any]]
2675diff --git a/certbot/plugins/enhancements_test.py b/certbot/plugins/enhancements_test.py
2676new file mode 100644
2677index 0000000..b69dc98
2678--- /dev/null
2679+++ b/certbot/plugins/enhancements_test.py
2680@@ -0,0 +1,65 @@
2681+"""Tests for new style enhancements"""
2682+import unittest
2683+import mock
2684+
2685+from certbot.plugins import enhancements
2686+from certbot.plugins import null
2687+
2688+import certbot.tests.util as test_util
2689+
2690+
2691+class EnhancementTest(test_util.ConfigTestCase):
2692+ """Tests for new style enhancements in certbot.plugins.enhancements"""
2693+
2694+ def setUp(self):
2695+ super(EnhancementTest, self).setUp()
2696+ self.mockinstaller = mock.MagicMock(spec=enhancements.AutoHSTSEnhancement)
2697+
2698+
2699+ @test_util.patch_get_utility()
2700+ def test_enhancement_enabled_enhancements(self, _):
2701+ FAKEINDEX = [
2702+ {
2703+ "name": "autohsts",
2704+ "cli_dest": "auto_hsts",
2705+ },
2706+ {
2707+ "name": "somethingelse",
2708+ "cli_dest": "something",
2709+ }
2710+ ]
2711+ with mock.patch("certbot.plugins.enhancements._INDEX", FAKEINDEX):
2712+ self.config.auto_hsts = True
2713+ self.config.something = True
2714+ enabled = list(enhancements.enabled_enhancements(self.config))
2715+ self.assertEqual(len(enabled), 2)
2716+ self.assertTrue([i for i in enabled if i["name"] == "autohsts"])
2717+ self.assertTrue([i for i in enabled if i["name"] == "somethingelse"])
2718+
2719+ def test_are_requested(self):
2720+ self.assertEquals(
2721+ len([i for i in enhancements.enabled_enhancements(self.config)]), 0)
2722+ self.assertFalse(enhancements.are_requested(self.config))
2723+ self.config.auto_hsts = True
2724+ self.assertEquals(
2725+ len([i for i in enhancements.enabled_enhancements(self.config)]), 1)
2726+ self.assertTrue(enhancements.are_requested(self.config))
2727+
2728+ def test_are_supported(self):
2729+ self.config.auto_hsts = True
2730+ unsupported = null.Installer(self.config, "null")
2731+ self.assertTrue(enhancements.are_supported(self.config, self.mockinstaller))
2732+ self.assertFalse(enhancements.are_supported(self.config, unsupported))
2733+
2734+ def test_enable(self):
2735+ self.config.auto_hsts = True
2736+ domains = ["example.com", "www.example.com"]
2737+ lineage = "lineage"
2738+ enhancements.enable(lineage, domains, self.mockinstaller, self.config)
2739+ self.assertTrue(self.mockinstaller.enable_autohsts.called)
2740+ self.assertEquals(self.mockinstaller.enable_autohsts.call_args[0],
2741+ (lineage, domains))
2742+
2743+
2744+if __name__ == '__main__':
2745+ unittest.main() # pragma: no cover
2746diff --git a/certbot/plugins/manual.py b/certbot/plugins/manual.py
2747index 614449d..53533d3 100644
2748--- a/certbot/plugins/manual.py
2749+++ b/certbot/plugins/manual.py
2750@@ -5,7 +5,9 @@ import zope.component
2751 import zope.interface
2752
2753 from acme import challenges
2754+from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
2755
2756+from certbot import achallenges # pylint: disable=unused-import
2757 from certbot import interfaces
2758 from certbot import errors
2759 from certbot import hooks
2760@@ -98,7 +100,8 @@ when it receives a TLS ClientHello with the SNI extension set to
2761 super(Authenticator, self).__init__(*args, **kwargs)
2762 self.reverter = reverter.Reverter(self.config)
2763 self.reverter.recovery_routine()
2764- self.env = dict()
2765+ self.env = dict() \
2766+ # type: Dict[achallenges.KeyAuthorizationAnnotatedChallenge, Dict[str, str]]
2767 self.tls_sni_01 = None
2768
2769 @classmethod
2770diff --git a/certbot/plugins/selection.py b/certbot/plugins/selection.py
2771index 5b19531..9c21382 100644
2772--- a/certbot/plugins/selection.py
2773+++ b/certbot/plugins/selection.py
2774@@ -39,6 +39,35 @@ def pick_authenticator(
2775 return pick_plugin(
2776 config, default, plugins, question, (interfaces.IAuthenticator,))
2777
2778+def get_unprepared_installer(config, plugins):
2779+ """
2780+ Get an unprepared interfaces.IInstaller object.
2781+
2782+ :param certbot.interfaces.IConfig config: Configuration
2783+ :param certbot.plugins.disco.PluginsRegistry plugins:
2784+ All plugins registered as entry points.
2785+
2786+ :returns: Unprepared installer plugin or None
2787+ :rtype: IPlugin or None
2788+ """
2789+
2790+ _, req_inst = cli_plugin_requests(config)
2791+ if not req_inst:
2792+ return None
2793+ installers = plugins.filter(lambda p_ep: p_ep.name == req_inst)
2794+ installers.init(config)
2795+ installers = installers.verify((interfaces.IInstaller,))
2796+ if len(installers) > 1:
2797+ raise errors.PluginSelectionError(
2798+ "Found multiple installers with the name %s, Certbot is unable to "
2799+ "determine which one to use. Skipping." % req_inst)
2800+ if installers:
2801+ inst = list(installers.values())[0]
2802+ logger.debug("Selecting plugin: %s", inst)
2803+ return inst.init(config)
2804+ else:
2805+ raise errors.PluginSelectionError(
2806+ "Could not select or initialize the requested installer %s." % req_inst)
2807
2808 def pick_plugin(config, default, plugins, question, ifaces):
2809 """Pick plugin.
2810@@ -135,18 +164,20 @@ def choose_plugin(prepared, question):
2811 return None
2812
2813 noninstaller_plugins = ["webroot", "manual", "standalone", "dns-cloudflare", "dns-cloudxns",
2814- "dns-digitalocean", "dns-dnsimple", "dns-dnsmadeeasy", "dns-google",
2815- "dns-luadns", "dns-nsone", "dns-rfc2136", "dns-route53"]
2816+ "dns-digitalocean", "dns-dnsimple", "dns-dnsmadeeasy", "dns-gehirn",
2817+ "dns-google", "dns-linode", "dns-luadns", "dns-nsone", "dns-ovh",
2818+ "dns-rfc2136", "dns-route53", "dns-sakuracloud"]
2819
2820 def record_chosen_plugins(config, plugins, auth, inst):
2821 "Update the config entries to reflect the plugins we actually selected."
2822- config.authenticator = plugins.find_init(auth).name if auth else "None"
2823- config.installer = plugins.find_init(inst).name if inst else "None"
2824+ config.authenticator = plugins.find_init(auth).name if auth else None
2825+ config.installer = plugins.find_init(inst).name if inst else None
2826 logger.info("Plugins selected: Authenticator %s, Installer %s",
2827 config.authenticator, config.installer)
2828
2829
2830 def choose_configurator_plugins(config, plugins, verb):
2831+ # pylint: disable=too-many-branches
2832 """
2833 Figure out which configurator we're going to use, modifies
2834 config.authenticator and config.installer strings to reflect that choice if
2835@@ -159,6 +190,11 @@ def choose_configurator_plugins(config, plugins, verb):
2836 """
2837
2838 req_auth, req_inst = cli_plugin_requests(config)
2839+ installer_question = None
2840+
2841+ if verb == "enhance":
2842+ installer_question = ("Which installer would you like to use to "
2843+ "configure the selected enhancements?")
2844
2845 # Which plugins do we need?
2846 if verb == "run":
2847@@ -176,11 +212,11 @@ def choose_configurator_plugins(config, plugins, verb):
2848 need_inst = need_auth = False
2849 if verb == "certonly":
2850 need_auth = True
2851- if verb == "install":
2852+ if verb == "install" or verb == "enhance":
2853 need_inst = True
2854 if config.authenticator:
2855- logger.warning("Specifying an authenticator doesn't make sense in install mode")
2856-
2857+ logger.warning("Specifying an authenticator doesn't make sense when "
2858+ "running Certbot with verb \"%s\"", verb)
2859 # Try to meet the user's request and/or ask them to pick plugins
2860 authenticator = installer = None
2861 if verb == "run" and req_auth == req_inst:
2862@@ -189,7 +225,7 @@ def choose_configurator_plugins(config, plugins, verb):
2863 authenticator = installer = pick_configurator(config, req_inst, plugins)
2864 else:
2865 if need_inst or req_inst:
2866- installer = pick_installer(config, req_inst, plugins)
2867+ installer = pick_installer(config, req_inst, plugins, installer_question)
2868 if need_auth:
2869 authenticator = pick_authenticator(config, req_auth, plugins)
2870 logger.debug("Selected authenticator %s and installer %s", authenticator, installer)
2871@@ -253,16 +289,24 @@ def cli_plugin_requests(config): # pylint: disable=too-many-branches
2872 req_auth = set_configurator(req_auth, "dns-dnsimple")
2873 if config.dns_dnsmadeeasy:
2874 req_auth = set_configurator(req_auth, "dns-dnsmadeeasy")
2875+ if config.dns_gehirn:
2876+ req_auth = set_configurator(req_auth, "dns-gehirn")
2877 if config.dns_google:
2878 req_auth = set_configurator(req_auth, "dns-google")
2879+ if config.dns_linode:
2880+ req_auth = set_configurator(req_auth, "dns-linode")
2881 if config.dns_luadns:
2882 req_auth = set_configurator(req_auth, "dns-luadns")
2883 if config.dns_nsone:
2884 req_auth = set_configurator(req_auth, "dns-nsone")
2885+ if config.dns_ovh:
2886+ req_auth = set_configurator(req_auth, "dns-ovh")
2887 if config.dns_rfc2136:
2888 req_auth = set_configurator(req_auth, "dns-rfc2136")
2889 if config.dns_route53:
2890 req_auth = set_configurator(req_auth, "dns-route53")
2891+ if config.dns_sakuracloud:
2892+ req_auth = set_configurator(req_auth, "dns-sakuracloud")
2893 logger.debug("Requested authenticator %s and installer %s", req_auth, req_inst)
2894 return req_auth, req_inst
2895
2896diff --git a/certbot/plugins/selection_test.py b/certbot/plugins/selection_test.py
2897index 4112810..44d64ab 100644
2898--- a/certbot/plugins/selection_test.py
2899+++ b/certbot/plugins/selection_test.py
2900@@ -6,9 +6,13 @@ import unittest
2901 import mock
2902 import zope.component
2903
2904+from certbot import errors
2905+from certbot import interfaces
2906+
2907+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
2908 from certbot.display import util as display_util
2909+from certbot.plugins.disco import PluginsRegistry
2910 from certbot.tests import util as test_util
2911-from certbot import interfaces
2912
2913
2914 class ConveniencePickPluginTest(unittest.TestCase):
2915@@ -47,7 +51,7 @@ class PickPluginTest(unittest.TestCase):
2916 self.default = None
2917 self.reg = mock.MagicMock()
2918 self.question = "Question?"
2919- self.ifaces = []
2920+ self.ifaces = [] # type: List[interfaces.IPlugin]
2921
2922 def _call(self):
2923 from certbot.plugins.selection import pick_plugin
2924@@ -169,5 +173,48 @@ class ChoosePluginTest(unittest.TestCase):
2925
2926 self.assertTrue("default" in mock_util().menu.call_args[1])
2927
2928+class GetUnpreparedInstallerTest(test_util.ConfigTestCase):
2929+ """Tests for certbot.plugins.selection.get_unprepared_installer."""
2930+
2931+ def setUp(self):
2932+ super(GetUnpreparedInstallerTest, self).setUp()
2933+ self.mock_apache_fail_ep = mock.Mock(
2934+ description_with_name="afail")
2935+ self.mock_apache_fail_ep.name = "afail"
2936+ self.mock_apache_ep = mock.Mock(
2937+ description_with_name="apache")
2938+ self.mock_apache_ep.name = "apache"
2939+ self.mock_apache_plugin = mock.MagicMock()
2940+ self.mock_apache_ep.init.return_value = self.mock_apache_plugin
2941+ self.plugins = PluginsRegistry({
2942+ "afail": self.mock_apache_fail_ep,
2943+ "apache": self.mock_apache_ep,
2944+ })
2945+
2946+ def _call(self):
2947+ from certbot.plugins.selection import get_unprepared_installer
2948+ return get_unprepared_installer(self.config, self.plugins)
2949+
2950+ def test_no_installer_defined(self):
2951+ self.config.configurator = None
2952+ self.assertEquals(self._call(), None)
2953+
2954+ def test_no_available_installers(self):
2955+ self.config.configurator = "apache"
2956+ self.plugins = PluginsRegistry({})
2957+ self.assertRaises(errors.PluginSelectionError, self._call)
2958+
2959+ def test_get_plugin(self):
2960+ self.config.configurator = "apache"
2961+ installer = self._call()
2962+ self.assertTrue(installer is self.mock_apache_plugin)
2963+
2964+ def test_multiple_installers_returned(self):
2965+ self.config.configurator = "apache"
2966+ # Two plugins with the same name
2967+ self.mock_apache_fail_ep.name = "apache"
2968+ self.assertRaises(errors.PluginSelectionError, self._call)
2969+
2970+
2971 if __name__ == "__main__":
2972 unittest.main() # pragma: no cover
2973diff --git a/certbot/plugins/standalone.py b/certbot/plugins/standalone.py
2974index 817403b..cb2e695 100644
2975--- a/certbot/plugins/standalone.py
2976+++ b/certbot/plugins/standalone.py
2977@@ -3,6 +3,8 @@ import argparse
2978 import collections
2979 import logging
2980 import socket
2981+# https://github.com/python/typeshed/blob/master/stdlib/2and3/socket.pyi
2982+from socket import errno as socket_errors # type: ignore
2983
2984 import OpenSSL
2985 import six
2986@@ -10,7 +12,10 @@ import zope.interface
2987
2988 from acme import challenges
2989 from acme import standalone as acme_standalone
2990+# pylint: disable=unused-import, no-name-in-module
2991+from acme.magic_typing import DefaultDict, Dict, Set, Tuple, List, Type, TYPE_CHECKING
2992
2993+from certbot import achallenges # pylint: disable=unused-import
2994 from certbot import errors
2995 from certbot import interfaces
2996
2997@@ -18,6 +23,11 @@ from certbot.plugins import common
2998
2999 logger = logging.getLogger(__name__)
3000
3001+if TYPE_CHECKING:
3002+ ServedType = DefaultDict[
3003+ acme_standalone.BaseDualNetworkedServers,
3004+ Set[achallenges.KeyAuthorizationAnnotatedChallenge]
3005+ ]
3006
3007 class ServerManager(object):
3008 """Standalone servers manager.
3009@@ -33,7 +43,7 @@ class ServerManager(object):
3010
3011 """
3012 def __init__(self, certs, http_01_resources):
3013- self._instances = {}
3014+ self._instances = {} # type: Dict[int, acme_standalone.BaseDualNetworkedServers]
3015 self.certs = certs
3016 self.http_01_resources = http_01_resources
3017
3018@@ -59,7 +69,8 @@ class ServerManager(object):
3019 address = (listenaddr, port)
3020 try:
3021 if challenge_type is challenges.TLSSNI01:
3022- servers = acme_standalone.TLSSNI01DualNetworkedServers(address, self.certs)
3023+ servers = acme_standalone.TLSSNI01DualNetworkedServers(
3024+ address, self.certs) # type: acme_standalone.BaseDualNetworkedServers
3025 else: # challenges.HTTP01
3026 servers = acme_standalone.HTTP01DualNetworkedServers(
3027 address, self.http_01_resources)
3028@@ -103,7 +114,8 @@ class ServerManager(object):
3029 return self._instances.copy()
3030
3031
3032-SUPPORTED_CHALLENGES = [challenges.TLSSNI01, challenges.HTTP01]
3033+SUPPORTED_CHALLENGES = [challenges.TLSSNI01, challenges.HTTP01] \
3034+# type: List[Type[challenges.KeyAuthorizationChallenge]]
3035
3036
3037 class SupportedChallengesAction(argparse.Action):
3038@@ -179,14 +191,15 @@ class Authenticator(common.Plugin):
3039 self.key = OpenSSL.crypto.PKey()
3040 self.key.generate_key(OpenSSL.crypto.TYPE_RSA, 2048)
3041
3042- self.served = collections.defaultdict(set)
3043+ self.served = collections.defaultdict(set) # type: ServedType
3044
3045 # Stuff below is shared across threads (i.e. servers read
3046 # values, main thread writes). Due to the nature of CPython's
3047 # GIL, the operations are safe, c.f.
3048 # https://docs.python.org/2/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe
3049- self.certs = {}
3050- self.http_01_resources = set()
3051+ self.certs = {} # type: Dict[bytes, Tuple[OpenSSL.crypto.PKey, OpenSSL.crypto.X509]]
3052+ self.http_01_resources = set() \
3053+ # type: Set[acme_standalone.HTTP01RequestHandler.HTTP01Resource]
3054
3055 self.servers = ServerManager(self.certs, self.http_01_resources)
3056
3057@@ -265,13 +278,13 @@ class Authenticator(common.Plugin):
3058
3059
3060 def _handle_perform_error(error):
3061- if error.socket_error.errno == socket.errno.EACCES:
3062+ if error.socket_error.errno == socket_errors.EACCES:
3063 raise errors.PluginError(
3064 "Could not bind TCP port {0} because you don't have "
3065 "the appropriate permissions (for example, you "
3066 "aren't running this program as "
3067 "root).".format(error.port))
3068- elif error.socket_error.errno == socket.errno.EADDRINUSE:
3069+ elif error.socket_error.errno == socket_errors.EADDRINUSE:
3070 display = zope.component.getUtility(interfaces.IDisplay)
3071 msg = (
3072 "Could not bind TCP port {0} because it is already in "
3073diff --git a/certbot/plugins/standalone_test.py b/certbot/plugins/standalone_test.py
3074index 5227bc5..47f44ff 100644
3075--- a/certbot/plugins/standalone_test.py
3076+++ b/certbot/plugins/standalone_test.py
3077@@ -2,12 +2,18 @@
3078 import argparse
3079 import socket
3080 import unittest
3081+# https://github.com/python/typeshed/blob/master/stdlib/2and3/socket.pyi
3082+from socket import errno as socket_errors # type: ignore
3083
3084 import josepy as jose
3085 import mock
3086 import six
3087
3088+import OpenSSL.crypto # pylint: disable=unused-import
3089+
3090 from acme import challenges
3091+from acme import standalone as acme_standalone # pylint: disable=unused-import
3092+from acme.magic_typing import Dict, Tuple, Set # pylint: disable=unused-import, no-name-in-module
3093
3094 from certbot import achallenges
3095 from certbot import errors
3096@@ -21,8 +27,9 @@ class ServerManagerTest(unittest.TestCase):
3097
3098 def setUp(self):
3099 from certbot.plugins.standalone import ServerManager
3100- self.certs = {}
3101- self.http_01_resources = {}
3102+ self.certs = {} # type: Dict[bytes, Tuple[OpenSSL.crypto.PKey, OpenSSL.crypto.X509]]
3103+ self.http_01_resources = {} \
3104+ # type: Set[acme_standalone.HTTP01RequestHandler.HTTP01Resource]
3105 self.mgr = ServerManager(self.certs, self.http_01_resources)
3106
3107 def test_init(self):
3108@@ -159,7 +166,7 @@ class AuthenticatorTest(unittest.TestCase):
3109 @test_util.patch_get_utility()
3110 def test_perform_eaddrinuse_retry(self, mock_get_utility):
3111 mock_utility = mock_get_utility()
3112- errno = socket.errno.EADDRINUSE
3113+ errno = socket_errors.EADDRINUSE
3114 error = errors.StandaloneBindError(mock.MagicMock(errno=errno), -1)
3115 self.auth.servers.run.side_effect = [error] + 2 * [mock.MagicMock()]
3116 mock_yesno = mock_utility.yesno
3117@@ -174,7 +181,7 @@ class AuthenticatorTest(unittest.TestCase):
3118 mock_yesno = mock_utility.yesno
3119 mock_yesno.return_value = False
3120
3121- errno = socket.errno.EADDRINUSE
3122+ errno = socket_errors.EADDRINUSE
3123 self.assertRaises(errors.PluginError, self._fail_perform, errno)
3124 self._assert_correct_yesno_call(mock_yesno)
3125
3126@@ -184,11 +191,11 @@ class AuthenticatorTest(unittest.TestCase):
3127 self.assertFalse(yesno_kwargs.get("default", True))
3128
3129 def test_perform_eacces(self):
3130- errno = socket.errno.EACCES
3131+ errno = socket_errors.EACCES
3132 self.assertRaises(errors.PluginError, self._fail_perform, errno)
3133
3134 def test_perform_unexpected_socket_error(self):
3135- errno = socket.errno.ENOTCONN
3136+ errno = socket_errors.ENOTCONN
3137 self.assertRaises(
3138 errors.StandaloneBindError, self._fail_perform, errno)
3139
3140diff --git a/certbot/plugins/storage.py b/certbot/plugins/storage.py
3141new file mode 100644
3142index 0000000..ae3ca18
3143--- /dev/null
3144+++ b/certbot/plugins/storage.py
3145@@ -0,0 +1,119 @@
3146+"""Plugin storage class."""
3147+import json
3148+import logging
3149+import os
3150+
3151+from acme.magic_typing import Any, Dict # pylint: disable=unused-import, no-name-in-module
3152+from certbot import errors
3153+
3154+logger = logging.getLogger(__name__)
3155+
3156+class PluginStorage(object):
3157+ """Class implementing storage functionality for plugins"""
3158+
3159+ def __init__(self, config, classkey):
3160+ """Initializes PluginStorage object storing required configuration
3161+ options.
3162+
3163+ :param .configuration.NamespaceConfig config: Configuration object
3164+ :param str classkey: class name to use as root key in storage file
3165+
3166+ """
3167+
3168+ self._config = config
3169+ self._classkey = classkey
3170+ self._initialized = False
3171+ self._data = None
3172+ self._storagepath = None
3173+
3174+ def _initialize_storage(self):
3175+ """Initializes PluginStorage data and reads current state from the disk
3176+ if the storage json exists."""
3177+
3178+ self._storagepath = os.path.join(self._config.config_dir, ".pluginstorage.json")
3179+ self._load()
3180+ self._initialized = True
3181+
3182+ def _load(self):
3183+ """Reads PluginStorage content from the disk to a dict structure
3184+
3185+ :raises .errors.PluginStorageError: when unable to open or read the file
3186+ """
3187+ data = dict() # type: Dict[str, Any]
3188+ filedata = ""
3189+ try:
3190+ with open(self._storagepath, 'r') as fh:
3191+ filedata = fh.read()
3192+ except IOError as e:
3193+ errmsg = "Could not read PluginStorage data file: {0} : {1}".format(
3194+ self._storagepath, str(e))
3195+ if os.path.isfile(self._storagepath):
3196+ # Only error out if file exists, but cannot be read
3197+ logger.error(errmsg)
3198+ raise errors.PluginStorageError(errmsg)
3199+ try:
3200+ data = json.loads(filedata)
3201+ except ValueError:
3202+ if not filedata:
3203+ logger.debug("Plugin storage file %s was empty, no values loaded",
3204+ self._storagepath)
3205+ else:
3206+ errmsg = "PluginStorage file {0} is corrupted.".format(
3207+ self._storagepath)
3208+ logger.error(errmsg)
3209+ raise errors.PluginStorageError(errmsg)
3210+ self._data = data
3211+
3212+ def save(self):
3213+ """Saves PluginStorage content to disk
3214+
3215+ :raises .errors.PluginStorageError: when unable to serialize the data
3216+ or write it to the filesystem
3217+ """
3218+ if not self._initialized:
3219+ errmsg = "Unable to save, no values have been added to PluginStorage."
3220+ logger.error(errmsg)
3221+ raise errors.PluginStorageError(errmsg)
3222+
3223+ try:
3224+ serialized = json.dumps(self._data)
3225+ except TypeError as e:
3226+ errmsg = "Could not serialize PluginStorage data: {0}".format(
3227+ str(e))
3228+ logger.error(errmsg)
3229+ raise errors.PluginStorageError(errmsg)
3230+ try:
3231+ with os.fdopen(os.open(self._storagepath,
3232+ os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
3233+ 0o600), 'w') as fh:
3234+ fh.write(serialized)
3235+ except IOError as e:
3236+ errmsg = "Could not write PluginStorage data to file {0} : {1}".format(
3237+ self._storagepath, str(e))
3238+ logger.error(errmsg)
3239+ raise errors.PluginStorageError(errmsg)
3240+
3241+ def put(self, key, value):
3242+ """Put configuration value to PluginStorage
3243+
3244+ :param str key: Key to store the value to
3245+ :param value: Data to store
3246+ """
3247+ if not self._initialized:
3248+ self._initialize_storage()
3249+
3250+ if not self._classkey in self._data.keys():
3251+ self._data[self._classkey] = dict()
3252+ self._data[self._classkey][key] = value
3253+
3254+ def fetch(self, key):
3255+ """Get configuration value from PluginStorage
3256+
3257+ :param str key: Key to get value from the storage
3258+
3259+ :raises KeyError: If the key doesn't exist in the storage
3260+ """
3261+ if not self._initialized:
3262+ self._initialize_storage()
3263+
3264+ return self._data[self._classkey][key]
3265diff --git a/certbot/plugins/storage_test.py b/certbot/plugins/storage_test.py
3266new file mode 100644
3267index 0000000..8d96f40
3268--- /dev/null
3269+++ b/certbot/plugins/storage_test.py
3270@@ -0,0 +1,117 @@
3271+"""Tests for certbot.plugins.storage.PluginStorage"""
3272+import json
3273+import mock
3274+import os
3275+import unittest
3276+
3277+from certbot import errors
3278+
3279+from certbot.plugins import common
3280+from certbot.tests import util as test_util
3281+
3282+class PluginStorageTest(test_util.ConfigTestCase):
3283+ """Test for certbot.plugins.storage.PluginStorage"""
3284+
3285+ def setUp(self):
3286+ super(PluginStorageTest, self).setUp()
3287+ self.plugin_cls = common.Installer
3288+ os.mkdir(self.config.config_dir)
3289+ with mock.patch("certbot.reverter.util"):
3290+ self.plugin = self.plugin_cls(config=self.config, name="mockplugin")
3291+
3292+ def test_load_errors_cant_read(self):
3293+ with open(os.path.join(self.config.config_dir,
3294+ ".pluginstorage.json"), "w") as fh:
3295+ fh.write("dummy")
3296+ # When unable to read file that exists
3297+ mock_open = mock.mock_open()
3298+ mock_open.side_effect = IOError
3299+ self.plugin.storage.storagepath = os.path.join(self.config.config_dir,
3300+ ".pluginstorage.json")
3301+ with mock.patch("six.moves.builtins.open", mock_open):
3302+ with mock.patch('os.path.isfile', return_value=True):
3303+ with mock.patch("certbot.reverter.util"):
3304+ self.assertRaises(errors.PluginStorageError,
3305+ self.plugin.storage._load) # pylint: disable=protected-access
3306+
3307+ def test_load_errors_empty(self):
3308+ with open(os.path.join(self.config.config_dir, ".pluginstorage.json"), "w") as fh:
3309+ fh.write('')
3310+ with mock.patch("certbot.plugins.storage.logger.debug") as mock_log:
3311+ # Should not error out but write a debug log line instead
3312+ with mock.patch("certbot.reverter.util"):
3313+ nocontent = self.plugin_cls(self.config, "mockplugin")
3314+ self.assertRaises(KeyError,
3315+ nocontent.storage.fetch, "value")
3316+ self.assertTrue(mock_log.called)
3317+ self.assertTrue("no values loaded" in mock_log.call_args[0][0])
3318+
3319+ def test_load_errors_corrupted(self):
3320+ with open(os.path.join(self.config.config_dir,
3321+ ".pluginstorage.json"), "w") as fh:
3322+ fh.write('invalid json')
3323+ with mock.patch("certbot.plugins.storage.logger.error") as mock_log:
3324+ with mock.patch("certbot.reverter.util"):
3325+ corrupted = self.plugin_cls(self.config, "mockplugin")
3326+ self.assertRaises(errors.PluginError,
3327+ corrupted.storage.fetch,
3328+ "value")
3329+ self.assertTrue("is corrupted" in mock_log.call_args[0][0])
3330+
3331+ def test_save_errors_cant_serialize(self):
3332+ with mock.patch("certbot.plugins.storage.logger.error") as mock_log:
3333+ # Set data as something that can't be serialized
3334+ self.plugin.storage._initialized = True # pylint: disable=protected-access
3335+ self.plugin.storage.storagepath = "/tmp/whatever"
3336+ self.plugin.storage._data = self.plugin_cls # pylint: disable=protected-access
3337+ self.assertRaises(errors.PluginStorageError,
3338+ self.plugin.storage.save)
3339+ self.assertTrue("Could not serialize" in mock_log.call_args[0][0])
3340+
3341+ def test_save_errors_unable_to_write_file(self):
3342+ mock_open = mock.mock_open()
3343+ mock_open.side_effect = IOError
3344+ with mock.patch("os.open", mock_open):
3345+ with mock.patch("certbot.plugins.storage.logger.error") as mock_log:
3346+ self.plugin.storage._data = {"valid": "data"} # pylint: disable=protected-access
3347+ self.plugin.storage._initialized = True # pylint: disable=protected-access
3348+ self.assertRaises(errors.PluginStorageError,
3349+ self.plugin.storage.save)
3350+ self.assertTrue("Could not write" in mock_log.call_args[0][0])
3351+
3352+ def test_save_uninitialized(self):
3353+ with mock.patch("certbot.reverter.util"):
3354+ self.assertRaises(errors.PluginStorageError,
3355+ self.plugin_cls(self.config, "x").storage.save)
3356+
3357+ def test_namespace_isolation(self):
3358+ with mock.patch("certbot.reverter.util"):
3359+ plugin1 = self.plugin_cls(self.config, "first")
3360+ plugin2 = self.plugin_cls(self.config, "second")
3361+ plugin1.storage.put("first_key", "first_value")
3362+ self.assertRaises(KeyError,
3363+ plugin2.storage.fetch, "first_key")
3364+ self.assertRaises(KeyError,
3365+ plugin2.storage.fetch, "first")
3366+ self.assertEqual(plugin1.storage.fetch("first_key"), "first_value")
3367+
3368+
3369+ def test_saved_state(self):
3370+ self.plugin.storage.put("testkey", "testvalue")
3371+ # Write to disk
3372+ self.plugin.storage.save()
3373+ with mock.patch("certbot.reverter.util"):
3374+ another = self.plugin_cls(self.config, "mockplugin")
3375+ self.assertEqual(another.storage.fetch("testkey"), "testvalue")
3376+
3377+ with open(os.path.join(self.config.config_dir,
3378+ ".pluginstorage.json"), 'r') as fh:
3379+ psdata = fh.read()
3380+ psjson = json.loads(psdata)
3381+ self.assertTrue("mockplugin" in psjson.keys())
3382+ self.assertEqual(len(psjson), 1)
3383+ self.assertEqual(psjson["mockplugin"]["testkey"], "testvalue")
3384+
3385+
3386+if __name__ == "__main__":
3387+ unittest.main() # pragma: no cover
3388diff --git a/certbot/plugins/util.py b/certbot/plugins/util.py
3389index ad2257e..3f03bf3 100644
3390--- a/certbot/plugins/util.py
3391+++ b/certbot/plugins/util.py
3392@@ -51,6 +51,6 @@ def path_surgery(cmd):
3393 return True
3394 else:
3395 expanded = " expanded" if any(added) else ""
3396- logger.warning("Failed to find executable %s in%s PATH: %s", cmd,
3397- expanded, path)
3398+ logger.debug("Failed to find executable %s in%s PATH: %s", cmd,
3399+ expanded, path)
3400 return False
3401diff --git a/certbot/plugins/util_test.py b/certbot/plugins/util_test.py
3402index 2c0e476..9757d8d 100644
3403--- a/certbot/plugins/util_test.py
3404+++ b/certbot/plugins/util_test.py
3405@@ -16,9 +16,8 @@ class GetPrefixTest(unittest.TestCase):
3406 class PathSurgeryTest(unittest.TestCase):
3407 """Tests for certbot.plugins.path_surgery."""
3408
3409- @mock.patch("certbot.plugins.util.logger.warning")
3410 @mock.patch("certbot.plugins.util.logger.debug")
3411- def test_path_surgery(self, mock_debug, mock_warn):
3412+ def test_path_surgery(self, mock_debug):
3413 from certbot.plugins.util import path_surgery
3414 all_path = {"PATH": "/usr/local/bin:/bin/:/usr/sbin/:/usr/local/sbin/"}
3415 with mock.patch.dict('os.environ', all_path):
3416@@ -26,14 +25,12 @@ class PathSurgeryTest(unittest.TestCase):
3417 mock_exists.return_value = True
3418 self.assertEqual(path_surgery("eg"), True)
3419 self.assertEqual(mock_debug.call_count, 0)
3420- self.assertEqual(mock_warn.call_count, 0)
3421 self.assertEqual(os.environ["PATH"], all_path["PATH"])
3422 no_path = {"PATH": "/tmp/"}
3423 with mock.patch.dict('os.environ', no_path):
3424 path_surgery("thingy")
3425- self.assertEqual(mock_debug.call_count, 1)
3426- self.assertEqual(mock_warn.call_count, 1)
3427- self.assertTrue("Failed to find" in mock_warn.call_args[0][0])
3428+ self.assertEqual(mock_debug.call_count, 2)
3429+ self.assertTrue("Failed to find" in mock_debug.call_args[0][0])
3430 self.assertTrue("/usr/local/bin" in os.environ["PATH"])
3431 self.assertTrue("/tmp" in os.environ["PATH"])
3432
3433diff --git a/certbot/plugins/webroot.py b/certbot/plugins/webroot.py
3434index 6328b16..5d0d7d5 100644
3435--- a/certbot/plugins/webroot.py
3436+++ b/certbot/plugins/webroot.py
3437@@ -10,8 +10,12 @@ import six
3438 import zope.component
3439 import zope.interface
3440
3441-from acme import challenges
3442+from acme import challenges # pylint: disable=unused-import
3443+# pylint: disable=unused-import, no-name-in-module
3444+from acme.magic_typing import Dict, Set, DefaultDict, List
3445+# pylint: enable=unused-import, no-name-in-module
3446
3447+from certbot import achallenges # pylint: disable=unused-import
3448 from certbot import cli
3449 from certbot import errors
3450 from certbot import interfaces
3451@@ -64,10 +68,11 @@ to serve all files under specified web root ({0})."""
3452
3453 def __init__(self, *args, **kwargs):
3454 super(Authenticator, self).__init__(*args, **kwargs)
3455- self.full_roots = {}
3456- self.performed = collections.defaultdict(set)
3457+ self.full_roots = {} # type: Dict[str, str]
3458+ self.performed = collections.defaultdict(set) \
3459+ # type: DefaultDict[str, Set[achallenges.KeyAuthorizationAnnotatedChallenge]]
3460 # stack of dirs successfully created by this authenticator
3461- self._created_dirs = []
3462+ self._created_dirs = [] # type: List[str]
3463
3464 def prepare(self): # pylint: disable=missing-docstring
3465 pass
3466@@ -156,7 +161,6 @@ to serve all files under specified web root ({0})."""
3467 " --help webroot for examples.")
3468 for name, path in path_map.items():
3469 self.full_roots[name] = os.path.join(path, challenges.HTTP01.URI_ROOT_PATH)
3470-
3471 logger.debug("Creating root challenges validation dir at %s",
3472 self.full_roots[name])
3473
3474@@ -207,7 +211,6 @@ to serve all files under specified web root ({0})."""
3475 os.umask(old_umask)
3476
3477 self.performed[root_path].add(achall)
3478-
3479 return response
3480
3481 def cleanup(self, achalls): # pylint: disable=missing-docstring
3482@@ -219,7 +222,7 @@ to serve all files under specified web root ({0})."""
3483 os.remove(validation_path)
3484 self.performed[root_path].remove(achall)
3485
3486- not_removed = []
3487+ not_removed = [] # type: List[str]
3488 while len(self._created_dirs) > 0:
3489 path = self._created_dirs.pop()
3490 try:
3491diff --git a/certbot/renewal.py b/certbot/renewal.py
3492index ea5d87a..ecc8b1f 100644
3493--- a/certbot/renewal.py
3494+++ b/certbot/renewal.py
3495@@ -11,14 +11,17 @@ import zope.component
3496
3497 import OpenSSL
3498
3499-from certbot import cli
3500+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
3501
3502+from certbot import cli
3503 from certbot import crypto_util
3504 from certbot import errors
3505 from certbot import interfaces
3506 from certbot import util
3507 from certbot import hooks
3508 from certbot import storage
3509+from certbot import updater
3510+
3511 from certbot.plugins import disco as plugins_disco
3512
3513 logger = logging.getLogger(__name__)
3514@@ -33,7 +36,8 @@ STR_CONFIG_ITEMS = ["config_dir", "logs_dir", "work_dir", "user_agent",
3515 "pre_hook", "post_hook", "tls_sni_01_address",
3516 "http01_address"]
3517 INT_CONFIG_ITEMS = ["rsa_key_size", "tls_sni_01_port", "http01_port"]
3518-BOOL_CONFIG_ITEMS = ["must_staple", "allow_subset_of_names"]
3519+BOOL_CONFIG_ITEMS = ["must_staple", "allow_subset_of_names", "reuse_key",
3520+ "autorenew"]
3521
3522 CONFIG_ITEMS = set(itertools.chain(
3523 BOOL_CONFIG_ITEMS, INT_CONFIG_ITEMS, STR_CONFIG_ITEMS, ('pref_challs',)))
3524@@ -58,8 +62,8 @@ def _reconstitute(config, full_path):
3525 """
3526 try:
3527 renewal_candidate = storage.RenewableCert(full_path, config)
3528- except (errors.CertStorageError, IOError) as exc:
3529- logger.warning(exc)
3530+ except (errors.CertStorageError, IOError):
3531+ logger.warning("", exc_info=True)
3532 logger.warning("Renewal configuration file %s is broken. Skipping.", full_path)
3533 logger.debug("Traceback was:\n%s", traceback.format_exc())
3534 return None
3535@@ -132,14 +136,15 @@ def _restore_plugin_configs(config, renewalparams):
3536 # longer defined, stored copies of that parameter will be
3537 # deserialized as strings by this logic even if they were
3538 # originally meant to be some other type.
3539+ plugin_prefixes = [] # type: List[str]
3540 if renewalparams["authenticator"] == "webroot":
3541 _restore_webroot_config(config, renewalparams)
3542- plugin_prefixes = []
3543 else:
3544- plugin_prefixes = [renewalparams["authenticator"]]
3545+ plugin_prefixes.append(renewalparams["authenticator"])
3546
3547- if renewalparams.get("installer", None) is not None:
3548+ if renewalparams.get("installer") is not None:
3549 plugin_prefixes.append(renewalparams["installer"])
3550+
3551 for plugin_prefix in set(plugin_prefixes):
3552 plugin_prefix = plugin_prefix.replace('-', '_')
3553 for config_item, config_value in six.iteritems(renewalparams):
3554@@ -257,7 +262,7 @@ def should_renew(config, lineage):
3555 if config.renew_by_default:
3556 logger.debug("Auto-renewal forced with --force-renewal...")
3557 return True
3558- if lineage.should_autorenew(interactive=True):
3559+ if lineage.should_autorenew():
3560 logger.info("Cert is due for renewal, auto-renewing...")
3561 return True
3562 if config.dry_run:
3563@@ -294,7 +299,10 @@ def renew_cert(config, domains, le_client, lineage):
3564 _avoid_invalidating_lineage(config, lineage, original_server)
3565 if not domains:
3566 domains = lineage.names()
3567- new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains)
3568+ # The private key is the existing lineage private key if reuse_key is set.
3569+ # Otherwise, generate a fresh private key by passing None.
3570+ new_key = lineage.privkey if config.reuse_key else None
3571+ new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
3572 if config.dry_run:
3573 logger.debug("Dry run: skipping updating lineage at %s",
3574 os.path.dirname(lineage.cert))
3575@@ -315,13 +323,13 @@ def report(msgs, category):
3576 def _renew_describe_results(config, renew_successes, renew_failures,
3577 renew_skipped, parse_failures):
3578
3579- out = []
3580+ out = [] # type: List[str]
3581 notify = out.append
3582 disp = zope.component.getUtility(interfaces.IDisplay)
3583
3584 def notify_error(err):
3585 """Notify and log errors."""
3586- notify(err)
3587+ notify(str(err))
3588 logger.error(err)
3589
3590 if config.dry_run:
3591@@ -351,7 +359,7 @@ def _renew_describe_results(config, renew_successes, renew_failures,
3592 notify_error(report(renew_failures, "failure"))
3593
3594 if parse_failures:
3595- notify("\nAdditionally, the following renewal configuration files "
3596+ notify("\nAdditionally, the following renewal configurations "
3597 "were invalid: ")
3598 notify(report(parse_failures, "parsefail"))
3599
3600@@ -411,9 +419,9 @@ def handle_renewal_request(config):
3601 # XXX: ensure that each call here replaces the previous one
3602 zope.component.provideUtility(lineage_config)
3603 renewal_candidate.ensure_deployed()
3604+ from certbot import main
3605+ plugins = plugins_disco.PluginsRegistry.find_all()
3606 if should_renew(lineage_config, renewal_candidate):
3607- plugins = plugins_disco.PluginsRegistry.find_all()
3608- from certbot import main
3609 # domains have been restored into lineage_config by reconstitute
3610 # but they're unnecessary anyway because renew_cert here
3611 # will just grab them from the certificate
3612@@ -426,6 +434,10 @@ def handle_renewal_request(config):
3613 "cert", renewal_candidate.latest_common_version()))
3614 renew_skipped.append("%s expires on %s" % (renewal_candidate.fullchain,
3615 expiry.strftime("%Y-%m-%d")))
3616+ # Run updater interface methods
3617+ updater.run_generic_updaters(lineage_config, renewal_candidate,
3618+ plugins)
3619+
3620 except Exception as e: # pylint: disable=broad-except
3621 # obtain_cert (presumably) encountered an unanticipated problem.
3622 logger.warning("Attempting to renew cert (%s) from %s produced an "
3623diff --git a/certbot/reverter.py b/certbot/reverter.py
3624index 34feafc..683c0cc 100644
3625--- a/certbot/reverter.py
3626+++ b/certbot/reverter.py
3627@@ -82,8 +82,10 @@ class Reverter(object):
3628 self._recover_checkpoint(self.config.temp_checkpoint_dir)
3629 except errors.ReverterError:
3630 # We have a partial or incomplete recovery
3631- logger.fatal("Incomplete or failed recovery for %s",
3632- self.config.temp_checkpoint_dir)
3633+ logger.critical(
3634+ "Incomplete or failed recovery for %s",
3635+ self.config.temp_checkpoint_dir,
3636+ )
3637 raise errors.ReverterError("Unable to revert temporary config")
3638
3639 def rollback_checkpoints(self, rollback=1):
3640@@ -123,7 +125,7 @@ class Reverter(object):
3641 try:
3642 self._recover_checkpoint(cp_dir)
3643 except errors.ReverterError:
3644- logger.fatal("Failed to load checkpoint during rollback")
3645+ logger.critical("Failed to load checkpoint during rollback")
3646 raise errors.ReverterError(
3647 "Unable to load checkpoint during rollback")
3648 rollback -= 1
3649@@ -181,7 +183,7 @@ class Reverter(object):
3650 if for_logging:
3651 return os.linesep.join(output)
3652 zope.component.getUtility(interfaces.IDisplay).notification(
3653- os.linesep.join(output), force_interactive=True)
3654+ os.linesep.join(output), force_interactive=True, pause=False)
3655
3656 def add_to_temp_checkpoint(self, save_files, save_notes):
3657 """Add files to temporary checkpoint.
3658@@ -457,7 +459,7 @@ class Reverter(object):
3659 self._recover_checkpoint(self.config.in_progress_dir)
3660 except errors.ReverterError:
3661 # We have a partial or incomplete recovery
3662- logger.fatal("Incomplete or failed recovery for IN_PROGRESS "
3663+ logger.critical("Incomplete or failed recovery for IN_PROGRESS "
3664 "checkpoint - %s",
3665 self.config.in_progress_dir)
3666 raise errors.ReverterError(
3667@@ -494,7 +496,7 @@ class Reverter(object):
3668 "Certbot probably shut down unexpectedly",
3669 os.linesep, path)
3670 except (IOError, OSError):
3671- logger.fatal(
3672+ logger.critical(
3673 "Unable to remove filepaths contained within %s", file_list)
3674 raise errors.ReverterError(
3675 "Unable to remove filepaths contained within "
3676diff --git a/certbot/storage.py b/certbot/storage.py
3677index ed3922c..32d6771 100644
3678--- a/certbot/storage.py
3679+++ b/certbot/storage.py
3680@@ -239,10 +239,15 @@ def relevant_values(all_values):
3681 :rtype dict:
3682
3683 """
3684- return dict(
3685+ rv = dict(
3686 (option, value)
3687 for option, value in six.iteritems(all_values)
3688 if _relevant(option) and cli.option_was_set(option, value))
3689+ # We always save the server value to help with forward compatibility
3690+ # and behavioral consistency when versions of Certbot with different
3691+ # server defaults are used.
3692+ rv["server"] = all_values["server"]
3693+ return rv
3694
3695 def lineagename_for_filename(config_filename):
3696 """Returns the lineagename for a configuration filename.
3697@@ -920,10 +925,10 @@ class RenewableCert(object):
3698 :rtype: bool
3699
3700 """
3701- return ("autorenew" not in self.configuration or
3702- self.configuration.as_bool("autorenew"))
3703+ return ("autorenew" not in self.configuration["renewalparams"] or
3704+ self.configuration["renewalparams"].as_bool("autorenew"))
3705
3706- def should_autorenew(self, interactive=False):
3707+ def should_autorenew(self):
3708 """Should we now try to autorenew the most recent cert version?
3709
3710 This is a policy question and does not only depend on whether
3711@@ -934,16 +939,12 @@ class RenewableCert(object):
3712 Note that this examines the numerically most recent cert version,
3713 not the currently deployed version.
3714
3715- :param bool interactive: set to True to examine the question
3716- regardless of whether the renewal configuration allows
3717- automated renewal (for interactive use). Default False.
3718-
3719 :returns: whether an attempt should now be made to autorenew the
3720 most current cert version in this lineage
3721 :rtype: bool
3722
3723 """
3724- if interactive or self.autorenewal_is_enabled():
3725+ if self.autorenewal_is_enabled():
3726 # Consider whether to attempt to autorenew this cert now
3727
3728 # Renewals on the basis of revocation
3729@@ -1053,6 +1054,9 @@ class RenewableCert(object):
3730 "`cert.pem` : will break many server configurations, and "
3731 "should not be used\n"
3732 " without reading further documentation (see link below).\n\n"
3733+ "WARNING: DO NOT MOVE THESE FILES!\n"
3734+ " Certbot expects these files to remain in this location in order\n"
3735+ " to function properly!\n\n"
3736 "We recommend not moving these files. For more information, see the Certbot\n"
3737 "User Guide at https://certbot.eff.org/docs/using.html#where-are-my-"
3738 "certificates.\n")
3739diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py
3740index 8ebda56..b2be47d 100644
3741--- a/certbot/tests/account_test.py
3742+++ b/certbot/tests/account_test.py
3743@@ -95,6 +95,7 @@ class AccountMemoryStorageTest(unittest.TestCase):
3744
3745 class AccountFileStorageTest(test_util.ConfigTestCase):
3746 """Tests for certbot.account.AccountFileStorage."""
3747+ #pylint: disable=too-many-public-methods
3748
3749 def setUp(self):
3750 super(AccountFileStorageTest, self).setUp()
3751@@ -159,7 +160,8 @@ class AccountFileStorageTest(test_util.ConfigTestCase):
3752 self.assertEqual([], self.storage.find_all())
3753
3754 def test_find_all_load_skips(self):
3755- self.storage.load = mock.MagicMock(
3756+ # pylint: disable=protected-access
3757+ self.storage._load_for_server_path = mock.MagicMock(
3758 side_effect=["x", errors.AccountStorageError, "z"])
3759 with mock.patch("certbot.account.os.listdir") as mock_listdir:
3760 mock_listdir.return_value = ["x", "y", "z"]
3761@@ -175,6 +177,86 @@ class AccountFileStorageTest(test_util.ConfigTestCase):
3762 self.assertRaises(errors.AccountStorageError, self.storage.load,
3763 "x" + self.acc.id)
3764
3765+ def _set_server(self, server):
3766+ self.config.server = server
3767+ from certbot.account import AccountFileStorage
3768+ self.storage = AccountFileStorage(self.config)
3769+
3770+ def test_find_all_neither_exists(self):
3771+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3772+ self.assertEqual([], self.storage.find_all())
3773+ self.assertEqual([], self.storage.find_all())
3774+ self.assertFalse(os.path.islink(self.config.accounts_dir))
3775+
3776+ def test_find_all_find_before_save(self):
3777+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3778+ self.assertEqual([], self.storage.find_all())
3779+ self.storage.save(self.acc, self.mock_client)
3780+ self.assertEqual([self.acc], self.storage.find_all())
3781+ self.assertEqual([self.acc], self.storage.find_all())
3782+ self.assertFalse(os.path.islink(self.config.accounts_dir))
3783+ # we shouldn't have created a v1 account
3784+ prev_server_path = 'https://acme-staging.api.letsencrypt.org/directory'
3785+ self.assertFalse(os.path.isdir(self.config.accounts_dir_for_server_path(prev_server_path)))
3786+
3787+ def test_find_all_save_before_find(self):
3788+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3789+ self.storage.save(self.acc, self.mock_client)
3790+ self.assertEqual([self.acc], self.storage.find_all())
3791+ self.assertEqual([self.acc], self.storage.find_all())
3792+ self.assertFalse(os.path.islink(self.config.accounts_dir))
3793+ self.assertTrue(os.path.isdir(self.config.accounts_dir))
3794+ prev_server_path = 'https://acme-staging.api.letsencrypt.org/directory'
3795+ self.assertFalse(os.path.isdir(self.config.accounts_dir_for_server_path(prev_server_path)))
3796+
3797+ def test_find_all_server_downgrade(self):
3798+ # don't use v2 accounts with a v1 url
3799+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3800+ self.assertEqual([], self.storage.find_all())
3801+ self.storage.save(self.acc, self.mock_client)
3802+ self.assertEqual([self.acc], self.storage.find_all())
3803+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3804+ self.assertEqual([], self.storage.find_all())
3805+
3806+ def test_upgrade_version_staging(self):
3807+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3808+ self.storage.save(self.acc, self.mock_client)
3809+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3810+ self.assertEqual([self.acc], self.storage.find_all())
3811+
3812+ def test_upgrade_version_production(self):
3813+ self._set_server('https://acme-v01.api.letsencrypt.org/directory')
3814+ self.storage.save(self.acc, self.mock_client)
3815+ self._set_server('https://acme-v02.api.letsencrypt.org/directory')
3816+ self.assertEqual([self.acc], self.storage.find_all())
3817+
3818+ @mock.patch('os.rmdir')
3819+ def test_corrupted_account(self, mock_rmdir):
3820+ # pylint: disable=protected-access
3821+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3822+ self.storage.save(self.acc, self.mock_client)
3823+ mock_rmdir.side_effect = OSError
3824+ self.storage._load_for_server_path = mock.MagicMock(
3825+ side_effect=errors.AccountStorageError)
3826+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3827+ self.assertEqual([], self.storage.find_all())
3828+
3829+ def test_upgrade_load(self):
3830+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3831+ self.storage.save(self.acc, self.mock_client)
3832+ prev_account = self.storage.load(self.acc.id)
3833+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3834+ account = self.storage.load(self.acc.id)
3835+ self.assertEqual(prev_account, account)
3836+
3837+ def test_upgrade_load_single_account(self):
3838+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3839+ self.storage.save(self.acc, self.mock_client)
3840+ prev_account = self.storage.load(self.acc.id)
3841+ self._set_server_and_stop_symlink('https://acme-staging-v02.api.letsencrypt.org/directory')
3842+ account = self.storage.load(self.acc.id)
3843+ self.assertEqual(prev_account, account)
3844+
3845 def test_load_ioerror(self):
3846 self.storage.save(self.acc, self.mock_client)
3847 mock_open = mock.mock_open()
3848@@ -199,6 +281,52 @@ class AccountFileStorageTest(test_util.ConfigTestCase):
3849 def test_delete_no_account(self):
3850 self.assertRaises(errors.AccountNotFound, self.storage.delete, self.acc.id)
3851
3852+ def _assert_symlinked_account_removed(self):
3853+ # create v1 account
3854+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3855+ self.storage.save(self.acc, self.mock_client)
3856+ # ensure v2 isn't already linked to it
3857+ with mock.patch('certbot.constants.LE_REUSE_SERVERS', {}):
3858+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3859+ self.assertRaises(errors.AccountNotFound, self.storage.load, self.acc.id)
3860+
3861+ def _test_delete_folders(self, server_url):
3862+ # create symlinked servers
3863+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3864+ self.storage.save(self.acc, self.mock_client)
3865+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3866+ self.storage.load(self.acc.id)
3867+
3868+ # delete starting at given server_url
3869+ self._set_server(server_url)
3870+ self.storage.delete(self.acc.id)
3871+
3872+ # make sure we're gone from both urls
3873+ self._set_server('https://acme-staging.api.letsencrypt.org/directory')
3874+ self.assertRaises(errors.AccountNotFound, self.storage.load, self.acc.id)
3875+ self._set_server('https://acme-staging-v02.api.letsencrypt.org/directory')
3876+ self.assertRaises(errors.AccountNotFound, self.storage.load, self.acc.id)
3877+
3878+ def test_delete_folders_up(self):
3879+ self._test_delete_folders('https://acme-staging.api.letsencrypt.org/directory')
3880+ self._assert_symlinked_account_removed()
3881+
3882+ def test_delete_folders_down(self):
3883+ self._test_delete_folders('https://acme-staging-v02.api.letsencrypt.org/directory')
3884+ self._assert_symlinked_account_removed()
3885+
3886+ def _set_server_and_stop_symlink(self, server_path):
3887+ self._set_server(server_path)
3888+ with open(os.path.join(self.config.accounts_dir, 'foo'), 'w') as f:
3889+ f.write('bar')
3890+
3891+ def test_delete_shared_account_up(self):
3892+ self._set_server_and_stop_symlink('https://acme-staging-v02.api.letsencrypt.org/directory')
3893+ self._test_delete_folders('https://acme-staging.api.letsencrypt.org/directory')
3894+
3895+ def test_delete_shared_account_down(self):
3896+ self._set_server_and_stop_symlink('https://acme-staging-v02.api.letsencrypt.org/directory')
3897+ self._test_delete_folders('https://acme-staging-v02.api.letsencrypt.org/directory')
3898
3899 if __name__ == "__main__":
3900 unittest.main() # pragma: no cover
3901diff --git a/certbot/tests/auth_handler_test.py b/certbot/tests/auth_handler_test.py
3902index 9a8a134..76d1df9 100644
3903--- a/certbot/tests/auth_handler_test.py
3904+++ b/certbot/tests/auth_handler_test.py
3905@@ -10,6 +10,7 @@ import zope.component
3906 from acme import challenges
3907 from acme import client as acme_client
3908 from acme import messages
3909+from acme.magic_typing import Dict # pylint: disable=unused-import, no-name-in-module
3910
3911 from certbot import achallenges
3912 from certbot import errors
3913@@ -354,12 +355,13 @@ class PollChallengesTest(unittest.TestCase):
3914 acme_util.CHALLENGES, [messages.STATUS_PENDING] * 3, False), [])
3915 ]
3916
3917- self.chall_update = {}
3918+ self.chall_update = {} # type: Dict[int, achallenges.KeyAuthorizationAnnotatedChallenge]
3919 for i, aauthzr in enumerate(self.aauthzrs):
3920 self.chall_update[i] = [
3921 challb_to_achall(challb, mock.Mock(key="dummy_key"), self.doms[i])
3922 for challb in aauthzr.authzr.body.challenges]
3923
3924+
3925 @mock.patch("certbot.auth_handler.time")
3926 def test_poll_challenges(self, unused_mock_time):
3927 self.mock_net.poll.side_effect = self._mock_poll_solve_one_valid
3928diff --git a/certbot/tests/cert_manager_test.py b/certbot/tests/cert_manager_test.py
3929index 98ff163..6ec1d4f 100644
3930--- a/certbot/tests/cert_manager_test.py
3931+++ b/certbot/tests/cert_manager_test.py
3932@@ -216,11 +216,12 @@ class CertificatesTest(BaseCertManagerTest):
3933 cert.is_test_cert = False
3934 parsed_certs = [cert]
3935
3936+ mock_config = mock.MagicMock(certname=None, lineagename=None)
3937 # pylint: disable=protected-access
3938- get_report = lambda: cert_manager._report_human_readable(mock_config, parsed_certs)
3939
3940- mock_config = mock.MagicMock(certname=None, lineagename=None)
3941 # pylint: disable=protected-access
3942+ get_report = lambda: cert_manager._report_human_readable(mock_config, parsed_certs)
3943+
3944 out = get_report()
3945 self.assertTrue("INVALID: EXPIRED" in out)
3946
3947@@ -568,5 +569,103 @@ class MatchAndCheckOverlaps(storage_test.BaseRenewableCertTest):
3948 self.assertRaises(errors.OverlappingMatchFound, self._call, self.config, None, None, None)
3949
3950
3951+class GetCertnameTest(unittest.TestCase):
3952+ """Tests for certbot.cert_manager."""
3953+
3954+ def setUp(self):
3955+ self.get_utility_patch = test_util.patch_get_utility()
3956+ self.mock_get_utility = self.get_utility_patch.start()
3957+ self.config = mock.MagicMock()
3958+ self.config.certname = None
3959+
3960+ def tearDown(self):
3961+ self.get_utility_patch.stop()
3962+
3963+ @mock.patch('certbot.storage.renewal_conf_files')
3964+ @mock.patch('certbot.storage.lineagename_for_filename')
3965+ def test_get_certnames(self, mock_name, mock_files):
3966+ mock_files.return_value = ['example.com.conf']
3967+ mock_name.return_value = 'example.com'
3968+ from certbot import cert_manager
3969+ prompt = "Which certificate would you"
3970+ self.mock_get_utility().menu.return_value = (display_util.OK, 0)
3971+ self.assertEquals(
3972+ cert_manager.get_certnames(
3973+ self.config, "verb", allow_multiple=False), ['example.com'])
3974+ self.assertTrue(
3975+ prompt in self.mock_get_utility().menu.call_args[0][0])
3976+
3977+ @mock.patch('certbot.storage.renewal_conf_files')
3978+ @mock.patch('certbot.storage.lineagename_for_filename')
3979+ def test_get_certnames_custom_prompt(self, mock_name, mock_files):
3980+ mock_files.return_value = ['example.com.conf']
3981+ mock_name.return_value = 'example.com'
3982+ from certbot import cert_manager
3983+ prompt = "custom prompt"
3984+ self.mock_get_utility().menu.return_value = (display_util.OK, 0)
3985+ self.assertEquals(
3986+ cert_manager.get_certnames(
3987+ self.config, "verb", allow_multiple=False, custom_prompt=prompt),
3988+ ['example.com'])
3989+ self.assertEquals(self.mock_get_utility().menu.call_args[0][0],
3990+ prompt)
3991+
3992+ @mock.patch('certbot.storage.renewal_conf_files')
3993+ @mock.patch('certbot.storage.lineagename_for_filename')
3994+ def test_get_certnames_user_abort(self, mock_name, mock_files):
3995+ mock_files.return_value = ['example.com.conf']
3996+ mock_name.return_value = 'example.com'
3997+ from certbot import cert_manager
3998+ self.mock_get_utility().menu.return_value = (display_util.CANCEL, 0)
3999+ self.assertRaises(
4000+ errors.Error,
4001+ cert_manager.get_certnames,
4002+ self.config, "erroring_anyway", allow_multiple=False)
4003+
4004+ @mock.patch('certbot.storage.renewal_conf_files')
4005+ @mock.patch('certbot.storage.lineagename_for_filename')
4006+ def test_get_certnames_allow_multiple(self, mock_name, mock_files):
4007+ mock_files.return_value = ['example.com.conf']
4008+ mock_name.return_value = 'example.com'
4009+ from certbot import cert_manager
4010+ prompt = "Which certificate(s) would you"
4011+ self.mock_get_utility().checklist.return_value = (display_util.OK,
4012+ ['example.com'])
4013+ self.assertEquals(
4014+ cert_manager.get_certnames(
4015+ self.config, "verb", allow_multiple=True), ['example.com'])
4016+ self.assertTrue(
4017+ prompt in self.mock_get_utility().checklist.call_args[0][0])
4018+
4019+ @mock.patch('certbot.storage.renewal_conf_files')
4020+ @mock.patch('certbot.storage.lineagename_for_filename')
4021+ def test_get_certnames_allow_multiple_custom_prompt(self, mock_name, mock_files):
4022+ mock_files.return_value = ['example.com.conf']
4023+ mock_name.return_value = 'example.com'
4024+ from certbot import cert_manager
4025+ prompt = "custom prompt"
4026+ self.mock_get_utility().checklist.return_value = (display_util.OK,
4027+ ['example.com'])
4028+ self.assertEquals(
4029+ cert_manager.get_certnames(
4030+ self.config, "verb", allow_multiple=True, custom_prompt=prompt),
4031+ ['example.com'])
4032+ self.assertEquals(
4033+ self.mock_get_utility().checklist.call_args[0][0],
4034+ prompt)
4035+
4036+ @mock.patch('certbot.storage.renewal_conf_files')
4037+ @mock.patch('certbot.storage.lineagename_for_filename')
4038+ def test_get_certnames_allow_multiple_user_abort(self, mock_name, mock_files):
4039+ mock_files.return_value = ['example.com.conf']
4040+ mock_name.return_value = 'example.com'
4041+ from certbot import cert_manager
4042+ self.mock_get_utility().checklist.return_value = (display_util.CANCEL, [])
4043+ self.assertRaises(
4044+ errors.Error,
4045+ cert_manager.get_certnames,
4046+ self.config, "erroring_anyway", allow_multiple=True)
4047+
4048+
4049 if __name__ == "__main__":
4050 unittest.main() # pragma: no cover
4051diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py
4052index 1bba699..979cd97 100644
4053--- a/certbot/tests/cli_test.py
4054+++ b/certbot/tests/cli_test.py
4055@@ -495,7 +495,8 @@ class SetByCliTest(unittest.TestCase):
4056 for v in ('manual', 'manual_auth_hook', 'manual_public_ip_logging_ok'):
4057 self.assertTrue(_call_set_by_cli(v, args, verb))
4058
4059- cli.set_by_cli.detector = None
4060+ # https://github.com/python/mypy/issues/2087
4061+ cli.set_by_cli.detector = None # type: ignore
4062
4063 args = ['--manual-auth-hook', 'command']
4064 for v in ('manual_auth_hook', 'manual_public_ip_logging_ok'):
4065diff --git a/certbot/tests/client_test.py b/certbot/tests/client_test.py
4066index 0f2c581..70ab4c7 100644
4067--- a/certbot/tests/client_test.py
4068+++ b/certbot/tests/client_test.py
4069@@ -1,5 +1,6 @@
4070 """Tests for certbot.client."""
4071 import os
4072+import platform
4073 import shutil
4074 import tempfile
4075 import unittest
4076@@ -12,11 +13,42 @@ from certbot import util
4077
4078 import certbot.tests.util as test_util
4079
4080-
4081 KEY = test_util.load_vector("rsa512_key.pem")
4082 CSR_SAN = test_util.load_vector("csr-san_512.pem")
4083
4084
4085+class DetermineUserAgentTest(test_util.ConfigTestCase):
4086+ """Tests for certbot.client.determine_user_agent."""
4087+
4088+ def _call(self):
4089+ from certbot.client import determine_user_agent
4090+ return determine_user_agent(self.config)
4091+
4092+ @mock.patch.dict(os.environ, {"CERTBOT_DOCS": "1"})
4093+ def test_docs_value(self):
4094+ self._test(expect_doc_values=True)
4095+
4096+ @mock.patch.dict(os.environ, {})
4097+ def test_real_values(self):
4098+ self._test(expect_doc_values=False)
4099+
4100+ def _test(self, expect_doc_values):
4101+ ua = self._call()
4102+
4103+ if expect_doc_values:
4104+ doc_value_check = self.assertIn
4105+ real_value_check = self.assertNotIn
4106+ else:
4107+ doc_value_check = self.assertNotIn
4108+ real_value_check = self.assertIn
4109+
4110+ doc_value_check("certbot(-auto)", ua)
4111+ doc_value_check("OS_NAME OS_VERSION", ua)
4112+ doc_value_check("major.minor.patchlevel", ua)
4113+ real_value_check(util.get_os_info_ua(), ua)
4114+ real_value_check(platform.python_version(), ua)
4115+
4116+
4117 class RegisterTest(test_util.ConfigTestCase):
4118 """Tests for certbot.client.register."""
4119
4120@@ -92,6 +124,20 @@ class RegisterTest(test_util.ConfigTestCase):
4121 mock_logger.info.assert_called_once_with(mock.ANY)
4122 self.assertTrue(mock_handle.called)
4123
4124+ @mock.patch("certbot.account.report_new_account")
4125+ @mock.patch("certbot.client.display_ops.get_email")
4126+ def test_dry_run_no_staging_account(self, _rep, mock_get_email):
4127+ """Tests dry-run for no staging account, expect account created with no email"""
4128+ with mock.patch("certbot.client.acme_client.BackwardsCompatibleClientV2") as mock_client:
4129+ with mock.patch("certbot.eff.handle_subscription"):
4130+ with mock.patch("certbot.account.report_new_account"):
4131+ self.config.dry_run = True
4132+ self._call()
4133+ # check Certbot did not ask the user to provide an email
4134+ self.assertFalse(mock_get_email.called)
4135+ # check Certbot created an account with no email. Contact should return empty
4136+ self.assertFalse(mock_client().new_account_and_tos.call_args[0][0].contact)
4137+
4138 def test_unsupported_error(self):
4139 from acme import messages
4140 msg = "Test"
4141@@ -105,6 +151,7 @@ class RegisterTest(test_util.ConfigTestCase):
4142
4143 class ClientTestCommon(test_util.ConfigTestCase):
4144 """Common base class for certbot.client.Client tests."""
4145+
4146 def setUp(self):
4147 super(ClientTestCommon, self).setUp()
4148 self.config.no_verify_ssl = False
4149@@ -124,6 +171,7 @@ class ClientTestCommon(test_util.ConfigTestCase):
4150
4151 class ClientTest(ClientTestCommon):
4152 """Tests for certbot.client.Client."""
4153+
4154 def setUp(self):
4155 super(ClientTest, self).setUp()
4156
4157@@ -286,10 +334,10 @@ class ClientTest(ClientTestCommon):
4158 @mock.patch('certbot.client.Client.obtain_certificate')
4159 @mock.patch('certbot.storage.RenewableCert.new_lineage')
4160 def test_obtain_and_enroll_certificate(self,
4161- mock_storage, mock_obtain_certificate):
4162+ mock_storage, mock_obtain_certificate):
4163 domains = ["*.example.com", "example.com"]
4164 mock_obtain_certificate.return_value = (mock.MagicMock(),
4165- mock.MagicMock(), mock.MagicMock(), None)
4166+ mock.MagicMock(), mock.MagicMock(), None)
4167
4168 self.client.config.dry_run = False
4169 self.assertTrue(self.client.obtain_and_enroll_certificate(domains, "example_cert"))
4170@@ -318,8 +366,8 @@ class ClientTest(ClientTestCommon):
4171 candidate_fullchain_path = os.path.join(tmp_path, "chains", "fullchain.pem")
4172 mock_parser.verb = "certonly"
4173 mock_parser.args = ["--cert-path", candidate_cert_path,
4174- "--chain-path", candidate_chain_path,
4175- "--fullchain-path", candidate_fullchain_path]
4176+ "--chain-path", candidate_chain_path,
4177+ "--fullchain-path", candidate_fullchain_path]
4178
4179 cert_path, chain_path, fullchain_path = self.client.save_certificate(
4180 cert_pem, chain_pem, candidate_cert_path, candidate_chain_path,
4181@@ -407,6 +455,7 @@ class ClientTest(ClientTestCommon):
4182
4183 class EnhanceConfigTest(ClientTestCommon):
4184 """Tests for certbot.client.Client.enhance_config."""
4185+
4186 def setUp(self):
4187 super(EnhanceConfigTest, self).setUp()
4188
4189@@ -433,6 +482,22 @@ class EnhanceConfigTest(ClientTestCommon):
4190 self.client.installer.enhance.assert_not_called()
4191 mock_enhancements.ask.assert_not_called()
4192
4193+ @mock.patch("certbot.client.logger")
4194+ def test_already_exists_header(self, mock_log):
4195+ self.config.hsts = True
4196+ self._test_with_already_existing()
4197+ self.assertTrue(mock_log.warning.called)
4198+ self.assertEquals(mock_log.warning.call_args[0][1],
4199+ 'Strict-Transport-Security')
4200+
4201+ @mock.patch("certbot.client.logger")
4202+ def test_already_exists_redirect(self, mock_log):
4203+ self.config.redirect = True
4204+ self._test_with_already_existing()
4205+ self.assertTrue(mock_log.warning.called)
4206+ self.assertEquals(mock_log.warning.call_args[0][1],
4207+ 'redirect')
4208+
4209 def test_no_ask_hsts(self):
4210 self.config.hsts = True
4211 self._test_with_all_supported()
4212@@ -508,6 +573,13 @@ class EnhanceConfigTest(ClientTestCommon):
4213 self.assertEqual(self.client.installer.save.call_count, 1)
4214 self.assertEqual(self.client.installer.restart.call_count, 1)
4215
4216+ def _test_with_already_existing(self):
4217+ self.client.installer = mock.MagicMock()
4218+ self.client.installer.supported_enhancements.return_value = [
4219+ "ensure-http-header", "redirect", "staple-ocsp"]
4220+ self.client.installer.enhance.side_effect = errors.PluginEnhancementAlreadyPresent()
4221+ self.client.enhance_config([self.domain], None)
4222+
4223
4224 class RollbackTest(unittest.TestCase):
4225 """Tests for certbot.client.rollback."""
4226diff --git a/certbot/tests/crypto_util_test.py b/certbot/tests/crypto_util_test.py
4227index 2fe0e3d..baf14b2 100644
4228--- a/certbot/tests/crypto_util_test.py
4229+++ b/certbot/tests/crypto_util_test.py
4230@@ -21,6 +21,9 @@ CERT_PATH = test_util.vector_path('cert_512.pem')
4231 CERT = test_util.load_vector('cert_512.pem')
4232 SS_CERT_PATH = test_util.vector_path('cert_2048.pem')
4233 SS_CERT = test_util.load_vector('cert_2048.pem')
4234+P256_KEY = test_util.load_vector('nistp256_key.pem')
4235+P256_CERT_PATH = test_util.vector_path('cert-nosans_nistp256.pem')
4236+P256_CERT = test_util.load_vector('cert-nosans_nistp256.pem')
4237
4238 class InitSaveKeyTest(test_util.TempDirTestCase):
4239 """Tests for certbot.crypto_util.init_save_key."""
4240@@ -217,6 +220,13 @@ class VerifyRenewableCertSigTest(VerifyCertSetup):
4241 def test_cert_sig_match(self):
4242 self.assertEqual(None, self._call(self.renewable_cert))
4243
4244+ def test_cert_sig_match_ec(self):
4245+ renewable_cert = mock.MagicMock()
4246+ renewable_cert.cert = P256_CERT_PATH
4247+ renewable_cert.chain = P256_CERT_PATH
4248+ renewable_cert.privkey = P256_KEY
4249+ self.assertEqual(None, self._call(renewable_cert))
4250+
4251 def test_cert_sig_mismatch(self):
4252 self.bad_renewable_cert.cert = test_util.vector_path('cert_512_bad.pem')
4253 self.assertRaises(errors.Error, self._call, self.bad_renewable_cert)
4254diff --git a/certbot/tests/display/completer_test.py b/certbot/tests/display/completer_test.py
4255index 333acf2..ac01103 100644
4256--- a/certbot/tests/display/completer_test.py
4257+++ b/certbot/tests/display/completer_test.py
4258@@ -8,6 +8,7 @@ import unittest
4259 import mock
4260 from six.moves import reload_module # pylint: disable=import-error
4261
4262+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
4263 from certbot.tests.util import TempDirTestCase
4264
4265 class CompleterTest(TempDirTestCase):
4266@@ -21,7 +22,7 @@ class CompleterTest(TempDirTestCase):
4267 if self.tempdir[-1] != os.sep:
4268 self.tempdir += os.sep
4269
4270- self.paths = []
4271+ self.paths = [] # type: List[str]
4272 # create some files and directories in temp_dir
4273 for c in string.ascii_lowercase:
4274 path = os.path.join(self.tempdir, c)
4275diff --git a/certbot/tests/display/ops_test.py b/certbot/tests/display/ops_test.py
4276index c4f58ba..9de8c5e 100644
4277--- a/certbot/tests/display/ops_test.py
4278+++ b/certbot/tests/display/ops_test.py
4279@@ -207,9 +207,9 @@ class ChooseNamesTest(unittest.TestCase):
4280 self.mock_install = mock.MagicMock()
4281
4282 @classmethod
4283- def _call(cls, installer):
4284+ def _call(cls, installer, question=None):
4285 from certbot.display.ops import choose_names
4286- return choose_names(installer)
4287+ return choose_names(installer, question)
4288
4289 @mock.patch("certbot.display.ops._choose_names_manually")
4290 def test_no_installer(self, mock_manual):
4291@@ -282,6 +282,15 @@ class ChooseNamesTest(unittest.TestCase):
4292 self.assertEqual(mock_util().checklist.call_count, 1)
4293
4294 @test_util.patch_get_utility("certbot.display.ops.z_util")
4295+ def test_filter_namees_override_question(self, mock_util):
4296+ self.mock_install.get_all_names.return_value = set(["example.com"])
4297+ mock_util().checklist.return_value = (display_util.OK, ["example.com"])
4298+ names = self._call(self.mock_install, "Custom")
4299+ self.assertEqual(names, ["example.com"])
4300+ self.assertEqual(mock_util().checklist.call_count, 1)
4301+ self.assertEqual(mock_util().checklist.call_args[0][0], "Custom")
4302+
4303+ @test_util.patch_get_utility("certbot.display.ops.z_util")
4304 def test_filter_names_nothing_selected(self, mock_util):
4305 self.mock_install.get_all_names.return_value = set(["example.com"])
4306 mock_util().checklist.return_value = (display_util.OK, [])
4307@@ -481,5 +490,42 @@ class ValidatorTests(unittest.TestCase):
4308 self.__validator, "msg", default="")
4309
4310
4311+class ChooseValuesTest(unittest.TestCase):
4312+ """Test choose_values."""
4313+ @classmethod
4314+ def _call(cls, values, question):
4315+ from certbot.display.ops import choose_values
4316+ return choose_values(values, question)
4317+
4318+ @test_util.patch_get_utility("certbot.display.ops.z_util")
4319+ def test_choose_names_success(self, mock_util):
4320+ items = ["first", "second", "third"]
4321+ mock_util().checklist.return_value = (display_util.OK, [items[2]])
4322+ result = self._call(items, None)
4323+ self.assertEquals(result, [items[2]])
4324+ self.assertTrue(mock_util().checklist.called)
4325+ self.assertEquals(mock_util().checklist.call_args[0][0], None)
4326+
4327+ @test_util.patch_get_utility("certbot.display.ops.z_util")
4328+ def test_choose_names_success_question(self, mock_util):
4329+ items = ["first", "second", "third"]
4330+ question = "Which one?"
4331+ mock_util().checklist.return_value = (display_util.OK, [items[1]])
4332+ result = self._call(items, question)
4333+ self.assertEquals(result, [items[1]])
4334+ self.assertTrue(mock_util().checklist.called)
4335+ self.assertEquals(mock_util().checklist.call_args[0][0], question)
4336+
4337+ @test_util.patch_get_utility("certbot.display.ops.z_util")
4338+ def test_choose_names_user_cancel(self, mock_util):
4339+ items = ["first", "second", "third"]
4340+ question = "Want to cancel?"
4341+ mock_util().checklist.return_value = (display_util.CANCEL, [])
4342+ result = self._call(items, question)
4343+ self.assertEquals(result, [])
4344+ self.assertTrue(mock_util().checklist.called)
4345+ self.assertEquals(mock_util().checklist.call_args[0][0], question)
4346+
4347+
4348 if __name__ == "__main__":
4349 unittest.main() # pragma: no cover
4350diff --git a/certbot/tests/eff_test.py b/certbot/tests/eff_test.py
4351index 160af19..8d0d577 100644
4352--- a/certbot/tests/eff_test.py
4353+++ b/certbot/tests/eff_test.py
4354@@ -1,4 +1,5 @@
4355 """Tests for certbot.eff."""
4356+import requests
4357 import unittest
4358
4359 import mock
4360@@ -118,11 +119,28 @@ class SubscribeTest(unittest.TestCase):
4361 @test_util.patch_get_utility()
4362 def test_not_ok(self, mock_get_utility):
4363 self.response.ok = False
4364+ self.response.raise_for_status.side_effect = requests.exceptions.HTTPError
4365 self._call() # pylint: disable=no-value-for-parameter
4366 actual = self._get_reported_message(mock_get_utility)
4367 unexpected_part = 'because'
4368 self.assertFalse(unexpected_part in actual)
4369
4370+ @test_util.patch_get_utility()
4371+ def test_response_not_json(self, mock_get_utility):
4372+ self.response.json.side_effect = ValueError()
4373+ self._call() # pylint: disable=no-value-for-parameter
4374+ actual = self._get_reported_message(mock_get_utility)
4375+ expected_part = 'problem'
4376+ self.assertTrue(expected_part in actual)
4377+
4378+ @test_util.patch_get_utility()
4379+ def test_response_json_missing_status_element(self, mock_get_utility):
4380+ self.json.clear()
4381+ self._call() # pylint: disable=no-value-for-parameter
4382+ actual = self._get_reported_message(mock_get_utility)
4383+ expected_part = 'problem'
4384+ self.assertTrue(expected_part in actual)
4385+
4386 def _get_reported_message(self, mock_get_utility):
4387 self.assertTrue(mock_get_utility().add_message.called)
4388 return mock_get_utility().add_message.call_args[0][0]
4389diff --git a/certbot/tests/error_handler_test.py b/certbot/tests/error_handler_test.py
4390index d4c48c2..a4a65e2 100644
4391--- a/certbot/tests/error_handler_test.py
4392+++ b/certbot/tests/error_handler_test.py
4393@@ -6,6 +6,9 @@ import sys
4394 import unittest
4395
4396 import mock
4397+# pylint: disable=unused-import, no-name-in-module
4398+from acme.magic_typing import Callable, Dict, Union
4399+# pylint: enable=unused-import, no-name-in-module
4400
4401
4402 def get_signals(signums):
4403@@ -23,8 +26,7 @@ def set_signals(sig_handler_dict):
4404 def signal_receiver(signums):
4405 """Context manager to catch signals"""
4406 signals = []
4407- prev_handlers = {}
4408- prev_handlers = get_signals(signums)
4409+ prev_handlers = get_signals(signums) # type: Dict[int, Union[int, None, Callable]]
4410 set_signals(dict((s, lambda s, _: signals.append(s)) for s in signums))
4411 yield signals
4412 set_signals(prev_handlers)
4413diff --git a/certbot/tests/hook_test.py b/certbot/tests/hook_test.py
4414index 8619a1a..c9cfc69 100644
4415--- a/certbot/tests/hook_test.py
4416+++ b/certbot/tests/hook_test.py
4417@@ -5,6 +5,7 @@ import unittest
4418
4419 import mock
4420
4421+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
4422 from certbot import errors
4423 from certbot.tests import util
4424
4425@@ -106,8 +107,8 @@ class PreHookTest(HookTest):
4426 super(PreHookTest, self).tearDown()
4427
4428 def _reset_pre_hook_already(self):
4429- from certbot.hooks import pre_hook
4430- pre_hook.already.clear()
4431+ from certbot.hooks import executed_pre_hooks
4432+ executed_pre_hooks.clear()
4433
4434 def test_certonly(self):
4435 self.config.verb = "certonly"
4436@@ -184,8 +185,8 @@ class PostHookTest(HookTest):
4437 super(PostHookTest, self).tearDown()
4438
4439 def _reset_post_hook_eventually(self):
4440- from certbot.hooks import post_hook
4441- post_hook.eventually = []
4442+ from certbot.hooks import post_hooks
4443+ del post_hooks[:]
4444
4445 def test_certonly_and_run_with_hook(self):
4446 for verb in ("certonly", "run",):
4447@@ -238,8 +239,8 @@ class PostHookTest(HookTest):
4448 self.assertEqual(self._get_eventually(), expected)
4449
4450 def _get_eventually(self):
4451- from certbot.hooks import post_hook
4452- return post_hook.eventually
4453+ from certbot.hooks import post_hooks
4454+ return post_hooks
4455
4456
4457 class RunSavedPostHooksTest(HookTest):
4458@@ -248,23 +249,23 @@ class RunSavedPostHooksTest(HookTest):
4459 @classmethod
4460 def _call(cls, *args, **kwargs):
4461 from certbot.hooks import run_saved_post_hooks
4462- return run_saved_post_hooks(*args, **kwargs)
4463+ return run_saved_post_hooks()
4464
4465 def _call_with_mock_execute_and_eventually(self, *args, **kwargs):
4466 """Call run_saved_post_hooks but mock out execute and eventually
4467
4468- certbot.hooks.post_hook.eventually is replaced with
4469+ certbot.hooks.post_hooks is replaced with
4470 self.eventually. The mock execute object is returned rather than
4471 the return value of run_saved_post_hooks.
4472
4473 """
4474- eventually_path = "certbot.hooks.post_hook.eventually"
4475+ eventually_path = "certbot.hooks.post_hooks"
4476 with mock.patch(eventually_path, new=self.eventually):
4477 return self._call_with_mock_execute(*args, **kwargs)
4478
4479 def setUp(self):
4480 super(RunSavedPostHooksTest, self).setUp()
4481- self.eventually = []
4482+ self.eventually = [] # type: List[str]
4483
4484 def test_empty(self):
4485 self.assertFalse(self._call_with_mock_execute_and_eventually().called)
4486diff --git a/certbot/tests/log_test.py b/certbot/tests/log_test.py
4487index 549d2c5..c599134 100644
4488--- a/certbot/tests/log_test.py
4489+++ b/certbot/tests/log_test.py
4490@@ -10,6 +10,7 @@ import mock
4491 import six
4492
4493 from acme import messages
4494+from acme.magic_typing import Optional # pylint: disable=unused-import, no-name-in-module
4495
4496 from certbot import constants
4497 from certbot import errors
4498@@ -21,9 +22,9 @@ class PreArgParseSetupTest(unittest.TestCase):
4499 """Tests for certbot.log.pre_arg_parse_setup."""
4500
4501 @classmethod
4502- def _call(cls, *args, **kwargs):
4503+ def _call(cls, *args, **kwargs): # pylint: disable=unused-argument
4504 from certbot.log import pre_arg_parse_setup
4505- return pre_arg_parse_setup(*args, **kwargs)
4506+ return pre_arg_parse_setup()
4507
4508 @mock.patch('certbot.log.sys')
4509 @mock.patch('certbot.log.pre_arg_parse_except_hook')
4510@@ -38,16 +39,16 @@ class PreArgParseSetupTest(unittest.TestCase):
4511 mock_root_logger.setLevel.assert_called_once_with(logging.DEBUG)
4512 self.assertEqual(mock_root_logger.addHandler.call_count, 2)
4513
4514- MemoryHandler = logging.handlers.MemoryHandler
4515- memory_handler = None
4516+ memory_handler = None # type: Optional[logging.handlers.MemoryHandler]
4517 for call in mock_root_logger.addHandler.call_args_list:
4518 handler = call[0][0]
4519- if memory_handler is None and isinstance(handler, MemoryHandler):
4520+ if memory_handler is None and isinstance(handler, logging.handlers.MemoryHandler):
4521 memory_handler = handler
4522+ target = memory_handler.target # type: ignore
4523 else:
4524 self.assertTrue(isinstance(handler, logging.StreamHandler))
4525 self.assertTrue(
4526- isinstance(memory_handler.target, logging.StreamHandler))
4527+ isinstance(target, logging.StreamHandler))
4528
4529 mock_register.assert_called_once_with(logging.shutdown)
4530 mock_sys.excepthook(1, 2, 3)
4531diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py
4532index b778f05..cc4e6c2 100644
4533--- a/certbot/tests/main_test.py
4534+++ b/certbot/tests/main_test.py
4535@@ -16,17 +16,22 @@ import josepy as jose
4536 import six
4537 from six.moves import reload_module # pylint: disable=import-error
4538
4539+from acme.magic_typing import List # pylint: disable=unused-import, no-name-in-module
4540 from certbot import account
4541 from certbot import cli
4542 from certbot import constants
4543 from certbot import configuration
4544 from certbot import crypto_util
4545 from certbot import errors
4546+from certbot import interfaces # pylint: disable=unused-import
4547 from certbot import main
4548+from certbot import updater
4549 from certbot import util
4550
4551 from certbot.plugins import disco
4552+from certbot.plugins import enhancements
4553 from certbot.plugins import manual
4554+from certbot.plugins import null
4555
4556 import certbot.tests.util as test_util
4557
4558@@ -49,10 +54,11 @@ class TestHandleIdenticalCerts(unittest.TestCase):
4559 self.assertEqual(ret, ("reinstall", mock_lineage))
4560
4561
4562-class RunTest(unittest.TestCase):
4563+class RunTest(test_util.ConfigTestCase):
4564 """Tests for certbot.main.run."""
4565
4566 def setUp(self):
4567+ super(RunTest, self).setUp()
4568 self.domain = 'example.org'
4569 self.patches = [
4570 mock.patch('certbot.main._get_and_save_cert'),
4571@@ -102,6 +108,15 @@ class RunTest(unittest.TestCase):
4572 self._call()
4573 self.mock_success_renewal.assert_called_once_with([self.domain])
4574
4575+ @mock.patch('certbot.main.plug_sel.choose_configurator_plugins')
4576+ def test_run_enhancement_not_supported(self, mock_choose):
4577+ mock_choose.return_value = (null.Installer(self.config, "null"), None)
4578+ plugins = disco.PluginsRegistry.find_all()
4579+ self.config.auto_hsts = True
4580+ self.assertRaises(errors.NotSupportedError,
4581+ main.run,
4582+ self.config, plugins)
4583+
4584
4585 class CertonlyTest(unittest.TestCase):
4586 """Tests for certbot.main.certonly."""
4587@@ -599,14 +614,14 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4588
4589 if mockisfile:
4590 orig_open = os.path.isfile
4591- def mock_isfile(fn, *args, **kwargs):
4592+ def mock_isfile(fn, *args, **kwargs): # pylint: disable=unused-argument
4593 """Mock os.path.isfile()"""
4594 if (fn.endswith("cert") or
4595 fn.endswith("chain") or
4596 fn.endswith("privkey")):
4597 return True
4598 else:
4599- return orig_open(fn, *args, **kwargs)
4600+ return orig_open(fn)
4601
4602 with mock.patch("os.path.isfile") as mock_if:
4603 mock_if.side_effect = mock_isfile
4604@@ -625,7 +640,8 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4605 toy_stdout = stdout if stdout else six.StringIO()
4606 with mock.patch('certbot.main.sys.stdout', new=toy_stdout):
4607 with mock.patch('certbot.main.sys.stderr') as stderr:
4608- ret = main.main(args[:]) # NOTE: parser can alter its args!
4609+ with mock.patch("certbot.util.atexit"):
4610+ ret = main.main(args[:]) # NOTE: parser can alter its args!
4611 return ret, toy_stdout, stderr
4612
4613 def test_no_flags(self):
4614@@ -747,18 +763,23 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4615 def test_installer_param_error(self, _inst, _rec):
4616 self.assertRaises(errors.ConfigurationError,
4617 self._call,
4618- ['install', '--key-path', '/tmp/key_path'])
4619- self.assertRaises(errors.ConfigurationError,
4620- self._call,
4621- ['install', '--cert-path', '/tmp/key_path'])
4622- self.assertRaises(errors.ConfigurationError,
4623- self._call,
4624- ['install'])
4625- self.assertRaises(errors.ConfigurationError,
4626- self._call,
4627 ['install', '--cert-name', 'notfound',
4628 '--key-path', 'invalid'])
4629
4630+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4631+ @mock.patch('certbot.main.plug_sel.pick_installer')
4632+ @mock.patch('certbot.cert_manager.get_certnames')
4633+ @mock.patch('certbot.main._install_cert')
4634+ def test_installer_select_cert(self, mock_inst, mock_getcert, _inst, _rec):
4635+ mock_lineage = mock.MagicMock(cert_path="/tmp/cert", chain_path="/tmp/chain",
4636+ fullchain_path="/tmp/chain",
4637+ key_path="/tmp/privkey")
4638+ with mock.patch("certbot.cert_manager.lineage_for_certname") as mock_getlin:
4639+ mock_getlin.return_value = mock_lineage
4640+ self._call(['install'], mockisfile=True)
4641+ self.assertTrue(mock_getcert.called)
4642+ self.assertTrue(mock_inst.called)
4643+
4644 @mock.patch('certbot.main._report_new_cert')
4645 @mock.patch('certbot.util.exe_exists')
4646 def test_configurator_selection(self, mock_exe_exists, unused_report):
4647@@ -834,7 +855,7 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4648 @mock.patch('certbot.main.plugins_disco')
4649 @mock.patch('certbot.main.cli.HelpfulArgumentParser.determine_help_topics')
4650 def test_plugins_no_args(self, _det, mock_disco):
4651- ifaces = []
4652+ ifaces = [] # type: List[interfaces.IPlugin]
4653 plugins = mock_disco.PluginsRegistry.find_all()
4654
4655 stdout = six.StringIO()
4656@@ -849,7 +870,7 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4657 @mock.patch('certbot.main.plugins_disco')
4658 @mock.patch('certbot.main.cli.HelpfulArgumentParser.determine_help_topics')
4659 def test_plugins_no_args_unprivileged(self, _det, mock_disco):
4660- ifaces = []
4661+ ifaces = [] # type: List[interfaces.IPlugin]
4662 plugins = mock_disco.PluginsRegistry.find_all()
4663
4664 def throw_error(directory, mode, uid, strict):
4665@@ -871,7 +892,7 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4666 @mock.patch('certbot.main.plugins_disco')
4667 @mock.patch('certbot.main.cli.HelpfulArgumentParser.determine_help_topics')
4668 def test_plugins_init(self, _det, mock_disco):
4669- ifaces = []
4670+ ifaces = [] # type: List[interfaces.IPlugin]
4671 plugins = mock_disco.PluginsRegistry.find_all()
4672
4673 stdout = six.StringIO()
4674@@ -889,7 +910,7 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4675 @mock.patch('certbot.main.plugins_disco')
4676 @mock.patch('certbot.main.cli.HelpfulArgumentParser.determine_help_topics')
4677 def test_plugins_prepare(self, _det, mock_disco):
4678- ifaces = []
4679+ ifaces = [] # type: List[interfaces.IPlugin]
4680 plugins = mock_disco.PluginsRegistry.find_all()
4681
4682 stdout = six.StringIO()
4683@@ -1022,8 +1043,9 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4684
4685 def _test_renewal_common(self, due_for_renewal, extra_args, log_out=None,
4686 args=None, should_renew=True, error_expected=False,
4687- quiet_mode=False, expiry_date=datetime.datetime.now()):
4688- # pylint: disable=too-many-locals,too-many-arguments
4689+ quiet_mode=False, expiry_date=datetime.datetime.now(),
4690+ reuse_key=False):
4691+ # pylint: disable=too-many-locals,too-many-arguments,too-many-branches
4692 cert_path = test_util.vector_path('cert_512.pem')
4693 chain_path = '/etc/letsencrypt/live/foo.bar/fullchain.pem'
4694 mock_lineage = mock.MagicMock(cert=cert_path, fullchain=chain_path,
4695@@ -1038,9 +1060,8 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4696 mock_client.obtain_certificate.return_value = (mock_certr, 'chain',
4697 mock_key, 'csr')
4698
4699- def write_msg(message, *args, **kwargs):
4700+ def write_msg(message, *args, **kwargs): # pylint: disable=unused-argument
4701 """Write message to stdout."""
4702- _, _ = args, kwargs
4703 stdout.write(message)
4704
4705 try:
4706@@ -1074,7 +1095,13 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4707 traceback.format_exc())
4708
4709 if should_renew:
4710- mock_client.obtain_certificate.assert_called_once_with(['isnot.org'])
4711+ if reuse_key:
4712+ # The location of the previous live privkey.pem is passed
4713+ # to obtain_certificate
4714+ mock_client.obtain_certificate.assert_called_once_with(['isnot.org'],
4715+ os.path.join(self.config.config_dir, "live/sample-renewal/privkey.pem"))
4716+ else:
4717+ mock_client.obtain_certificate.assert_called_once_with(['isnot.org'], None)
4718 else:
4719 self.assertEqual(mock_client.obtain_certificate.call_count, 0)
4720 except:
4721@@ -1124,6 +1151,17 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4722 args = ["renew", "--dry-run", "-tvv"]
4723 self._test_renewal_common(True, [], args=args, should_renew=True)
4724
4725+ def test_reuse_key(self):
4726+ test_util.make_lineage(self.config.config_dir, 'sample-renewal.conf')
4727+ args = ["renew", "--dry-run", "--reuse-key"]
4728+ self._test_renewal_common(True, [], args=args, should_renew=True, reuse_key=True)
4729+
4730+ @mock.patch('certbot.storage.RenewableCert.save_successor')
4731+ def test_reuse_key_no_dry_run(self, unused_save_successor):
4732+ test_util.make_lineage(self.config.config_dir, 'sample-renewal.conf')
4733+ args = ["renew", "--reuse-key"]
4734+ self._test_renewal_common(True, [], args=args, should_renew=True, reuse_key=True)
4735+
4736 @mock.patch('certbot.renewal.should_renew')
4737 def test_renew_skips_recent_certs(self, should_renew):
4738 should_renew.return_value = False
4739@@ -1232,7 +1270,9 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4740 self._test_renew_common(renewalparams=renewalparams, error_expected=True,
4741 names=names, assert_oc_called=False)
4742
4743- def test_renew_with_configurator(self):
4744+ @mock.patch('certbot.plugins.selection.choose_configurator_plugins')
4745+ def test_renew_with_configurator(self, mock_sel):
4746+ mock_sel.return_value = (mock.MagicMock(), mock.MagicMock())
4747 renewalparams = {'authenticator': 'webroot'}
4748 self._test_renew_common(
4749 renewalparams=renewalparams, assert_oc_called=True,
4750@@ -1430,7 +1470,9 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4751 mocked_storage = mock.MagicMock()
4752 mocked_account.AccountFileStorage.return_value = mocked_storage
4753 mocked_storage.find_all.return_value = ["an account"]
4754- mocked_det.return_value = (mock.MagicMock(), "foo")
4755+ mock_acc = mock.MagicMock()
4756+ mock_regr = mock_acc.regr
4757+ mocked_det.return_value = (mock_acc, "foo")
4758 cb_client = mock.MagicMock()
4759 mocked_client.Client.return_value = cb_client
4760 x = self._call_no_clientmock(
4761@@ -1440,14 +1482,29 @@ class MainTest(test_util.ConfigTestCase): # pylint: disable=too-many-public-met
4762 self.assertTrue(x[0] is None)
4763 # and we got supposedly did update the registration from
4764 # the server
4765- self.assertTrue(
4766- cb_client.acme.update_registration.called)
4767+ reg_arg = cb_client.acme.update_registration.call_args[0][0]
4768+ # Test the return value of .update() was used because
4769+ # the regr is immutable.
4770+ self.assertEqual(reg_arg, mock_regr.update())
4771 # and we saved the updated registration on disk
4772 self.assertTrue(mocked_storage.save_regr.called)
4773 self.assertTrue(
4774 email in mock_utility().add_message.call_args[0][0])
4775 self.assertTrue(mock_handle.called)
4776
4777+ @mock.patch('certbot.plugins.selection.choose_configurator_plugins')
4778+ @mock.patch('certbot.updater._run_updaters')
4779+ def test_plugin_selection_error(self, mock_run, mock_choose):
4780+ mock_choose.side_effect = errors.PluginSelectionError
4781+ self.assertRaises(errors.PluginSelectionError, main.renew_cert,
4782+ None, None, None)
4783+
4784+ self.config.dry_run = False
4785+ updater.run_generic_updaters(self.config, None, None)
4786+ # Make sure we're returning None, and hence not trying to run the
4787+ # without installer
4788+ self.assertFalse(mock_run.called)
4789+
4790
4791 class UnregisterTest(unittest.TestCase):
4792 def setUp(self):
4793@@ -1533,5 +1590,181 @@ class MakeOrVerifyNeededDirs(test_util.ConfigTestCase):
4794 strict=self.config.strict_permissions)
4795
4796
4797+class EnhanceTest(test_util.ConfigTestCase):
4798+ """Tests for certbot.main.enhance."""
4799+
4800+ def setUp(self):
4801+ super(EnhanceTest, self).setUp()
4802+ self.get_utility_patch = test_util.patch_get_utility()
4803+ self.mock_get_utility = self.get_utility_patch.start()
4804+ self.mockinstaller = mock.MagicMock(spec=enhancements.AutoHSTSEnhancement)
4805+
4806+ def tearDown(self):
4807+ self.get_utility_patch.stop()
4808+
4809+ def _call(self, args):
4810+ plugins = disco.PluginsRegistry.find_all()
4811+ config = configuration.NamespaceConfig(
4812+ cli.prepare_and_parse_args(plugins, args))
4813+
4814+ with mock.patch('certbot.cert_manager.get_certnames') as mock_certs:
4815+ mock_certs.return_value = ['example.com']
4816+ with mock.patch('certbot.cert_manager.domains_for_certname') as mock_dom:
4817+ mock_dom.return_value = ['example.com']
4818+ with mock.patch('certbot.main._init_le_client') as mock_init:
4819+ mock_client = mock.MagicMock()
4820+ mock_client.config = config
4821+ mock_init.return_value = mock_client
4822+ main.enhance(config, plugins)
4823+ return mock_client # returns the client
4824+
4825+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4826+ @mock.patch('certbot.cert_manager.lineage_for_certname')
4827+ @mock.patch('certbot.main.display_ops.choose_values')
4828+ @mock.patch('certbot.main._find_domains_or_certname')
4829+ def test_selection_question(self, mock_find, mock_choose, mock_lineage, _rec):
4830+ mock_lineage.return_value = mock.MagicMock(chain_path="/tmp/nonexistent")
4831+ mock_choose.return_value = ['example.com']
4832+ mock_find.return_value = (None, None)
4833+ with mock.patch('certbot.main.plug_sel.pick_installer') as mock_pick:
4834+ self._call(['enhance', '--redirect'])
4835+ self.assertTrue(mock_pick.called)
4836+ # Check that the message includes "enhancements"
4837+ self.assertTrue("enhancements" in mock_pick.call_args[0][3])
4838+
4839+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4840+ @mock.patch('certbot.cert_manager.lineage_for_certname')
4841+ @mock.patch('certbot.main.display_ops.choose_values')
4842+ @mock.patch('certbot.main._find_domains_or_certname')
4843+ def test_selection_auth_warning(self, mock_find, mock_choose, mock_lineage, _rec):
4844+ mock_lineage.return_value = mock.MagicMock(chain_path="/tmp/nonexistent")
4845+ mock_choose.return_value = ["example.com"]
4846+ mock_find.return_value = (None, None)
4847+ with mock.patch('certbot.main.plug_sel.pick_installer'):
4848+ with mock.patch('certbot.main.plug_sel.logger.warning') as mock_log:
4849+ mock_client = self._call(['enhance', '-a', 'webroot', '--redirect'])
4850+ self.assertTrue(mock_log.called)
4851+ self.assertTrue("make sense" in mock_log.call_args[0][0])
4852+ self.assertTrue(mock_client.enhance_config.called)
4853+
4854+ @mock.patch('certbot.cert_manager.lineage_for_certname')
4855+ @mock.patch('certbot.main.display_ops.choose_values')
4856+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4857+ def test_enhance_config_call(self, _rec, mock_choose, mock_lineage):
4858+ mock_lineage.return_value = mock.MagicMock(chain_path="/tmp/nonexistent")
4859+ mock_choose.return_value = ["example.com"]
4860+ with mock.patch('certbot.main.plug_sel.pick_installer'):
4861+ mock_client = self._call(['enhance', '--redirect', '--hsts'])
4862+ req_enh = ["redirect", "hsts"]
4863+ not_req_enh = ["uir"]
4864+ self.assertTrue(mock_client.enhance_config.called)
4865+ self.assertTrue(
4866+ all([getattr(mock_client.config, e) for e in req_enh]))
4867+ self.assertFalse(
4868+ any([getattr(mock_client.config, e) for e in not_req_enh]))
4869+ self.assertTrue(
4870+ "example.com" in mock_client.enhance_config.call_args[0][0])
4871+
4872+ @mock.patch('certbot.cert_manager.lineage_for_certname')
4873+ @mock.patch('certbot.main.display_ops.choose_values')
4874+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4875+ def test_enhance_noninteractive(self, _rec, mock_choose, mock_lineage):
4876+ mock_lineage.return_value = mock.MagicMock(
4877+ chain_path="/tmp/nonexistent")
4878+ mock_choose.return_value = ["example.com"]
4879+ with mock.patch('certbot.main.plug_sel.pick_installer'):
4880+ mock_client = self._call(['enhance', '--redirect',
4881+ '--hsts', '--non-interactive'])
4882+ self.assertTrue(mock_client.enhance_config.called)
4883+ self.assertFalse(mock_choose.called)
4884+
4885+ @mock.patch('certbot.main.display_ops.choose_values')
4886+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4887+ def test_user_abort_domains(self, _rec, mock_choose):
4888+ mock_choose.return_value = []
4889+ with mock.patch('certbot.main.plug_sel.pick_installer'):
4890+ self.assertRaises(errors.Error,
4891+ self._call,
4892+ ['enhance', '--redirect', '--hsts'])
4893+
4894+ def test_no_enhancements_defined(self):
4895+ self.assertRaises(errors.MisconfigurationError,
4896+ self._call, ['enhance', '-a', 'null'])
4897+
4898+ @mock.patch('certbot.main.plug_sel.choose_configurator_plugins')
4899+ @mock.patch('certbot.main.display_ops.choose_values')
4900+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4901+ def test_plugin_selection_error(self, _rec, mock_choose, mock_pick):
4902+ mock_choose.return_value = ["example.com"]
4903+ mock_pick.return_value = (None, None)
4904+ mock_pick.side_effect = errors.PluginSelectionError()
4905+ mock_client = self._call(['enhance', '--hsts'])
4906+ self.assertFalse(mock_client.enhance_config.called)
4907+
4908+ @mock.patch('certbot.cert_manager.lineage_for_certname')
4909+ @mock.patch('certbot.main.display_ops.choose_values')
4910+ @mock.patch('certbot.main.plug_sel.pick_installer')
4911+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4912+ @test_util.patch_get_utility()
4913+ def test_enhancement_enable(self, _, _rec, mock_inst, mock_choose, mock_lineage):
4914+ mock_inst.return_value = self.mockinstaller
4915+ mock_choose.return_value = ["example.com", "another.tld"]
4916+ mock_lineage.return_value = mock.MagicMock(chain_path="/tmp/nonexistent")
4917+ self._call(['enhance', '--auto-hsts'])
4918+ self.assertTrue(self.mockinstaller.enable_autohsts.called)
4919+ self.assertEquals(self.mockinstaller.enable_autohsts.call_args[0][1],
4920+ ["example.com", "another.tld"])
4921+
4922+ @mock.patch('certbot.cert_manager.lineage_for_certname')
4923+ @mock.patch('certbot.main.display_ops.choose_values')
4924+ @mock.patch('certbot.main.plug_sel.pick_installer')
4925+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4926+ @test_util.patch_get_utility()
4927+ def test_enhancement_enable_not_supported(self, _, _rec, mock_inst, mock_choose, mock_lineage):
4928+ mock_inst.return_value = null.Installer(self.config, "null")
4929+ mock_choose.return_value = ["example.com", "another.tld"]
4930+ mock_lineage.return_value = mock.MagicMock(chain_path="/tmp/nonexistent")
4931+ self.assertRaises(
4932+ errors.NotSupportedError,
4933+ self._call, ['enhance', '--auto-hsts'])
4934+
4935+ def test_enhancement_enable_conflict(self):
4936+ self.assertRaises(
4937+ errors.Error,
4938+ self._call, ['enhance', '--auto-hsts', '--hsts'])
4939+
4940+
4941+class InstallTest(test_util.ConfigTestCase):
4942+ """Tests for certbot.main.install."""
4943+
4944+ def setUp(self):
4945+ super(InstallTest, self).setUp()
4946+ self.mockinstaller = mock.MagicMock(spec=enhancements.AutoHSTSEnhancement)
4947+
4948+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4949+ @mock.patch('certbot.main.plug_sel.pick_installer')
4950+ def test_install_enhancement_not_supported(self, mock_inst, _rec):
4951+ mock_inst.return_value = null.Installer(self.config, "null")
4952+ plugins = disco.PluginsRegistry.find_all()
4953+ self.config.auto_hsts = True
4954+ self.config.certname = "nonexistent"
4955+ self.assertRaises(errors.NotSupportedError,
4956+ main.install,
4957+ self.config, plugins)
4958+
4959+ @mock.patch('certbot.main.plug_sel.record_chosen_plugins')
4960+ @mock.patch('certbot.main.plug_sel.pick_installer')
4961+ def test_install_enhancement_no_certname(self, mock_inst, _rec):
4962+ mock_inst.return_value = self.mockinstaller
4963+ plugins = disco.PluginsRegistry.find_all()
4964+ self.config.auto_hsts = True
4965+ self.config.certname = None
4966+ self.config.key_path = "/tmp/nonexistent"
4967+ self.config.cert_path = "/tmp/nonexistent"
4968+ self.assertRaises(errors.ConfigurationError,
4969+ main.install,
4970+ self.config, plugins)
4971+
4972+
4973 if __name__ == '__main__':
4974 unittest.main() # pragma: no cover
4975diff --git a/certbot/tests/renewupdater_test.py b/certbot/tests/renewupdater_test.py
4976new file mode 100644
4977index 0000000..5a36207
4978--- /dev/null
4979+++ b/certbot/tests/renewupdater_test.py
4980@@ -0,0 +1,125 @@
4981+"""Tests for renewal updater interfaces"""
4982+import unittest
4983+import mock
4984+
4985+from certbot import interfaces
4986+from certbot import main
4987+from certbot import updater
4988+
4989+from certbot.plugins import enhancements
4990+
4991+import certbot.tests.util as test_util
4992+
4993+
4994+class RenewUpdaterTest(test_util.ConfigTestCase):
4995+ """Tests for interfaces.RenewDeployer and interfaces.GenericUpdater"""
4996+
4997+ def setUp(self):
4998+ super(RenewUpdaterTest, self).setUp()
4999+ self.generic_updater = mock.MagicMock(spec=interfaces.GenericUpdater)
5000+ self.generic_updater.restart = mock.MagicMock()
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches

to status/vote changes: