Merge ~tonysimpson/snapstore-client:snapstore-library-integration into snapstore-client:master
- Git
- lp:~tonysimpson/snapstore-client
- snapstore-library-integration
- Merge into master
Status: | Work in progress |
---|---|
Proposed branch: | ~tonysimpson/snapstore-client:snapstore-library-integration |
Merge into: | snapstore-client:master |
Diff against target: |
3582 lines (+1951/-1049) 19 files modified
Makefile (+32/-12) dev/null (+0/-185) requirements.txt (+1/-0) snap/plugins/_python/__init__.py (+19/-0) snap/plugins/_python/_pip.py (+575/-0) snap/plugins/_python/_python_finder.py (+104/-0) snap/plugins/_python/_sitecustomize.py (+124/-0) snap/plugins/_python/errors.py (+79/-0) snap/plugins/python_ols.py (+499/-0) snap/snapcraft.yaml (+15/-8) snapstore (+1/-126) snapstore_client/__main__.py (+136/-0) snapstore_client/logic/login.py (+36/-59) snapstore_client/logic/overrides.py (+79/-71) snapstore_client/logic/push.py (+0/-5) snapstore_client/logic/tests/test_login.py (+103/-265) snapstore_client/logic/tests/test_overrides.py (+106/-313) snapstore_client/logic/tests/utils.py (+42/-0) snapstore_client/utils.py (+0/-5) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Simon Davy (community) | Approve | ||
Review via email: mp+371039@code.launchpad.net |
Commit message
Integrate snapstore_library auth and overrides
Description of the change
Integrates the snapstore-library authentication and override
Tests have been simplified because snapstore-library has test coverage. Updated tests just cover functionality in CLI code.
snap/plugins/* should be ignore as it is just a copy of code from lp:snapstore-snap
There is a snapstore-library branch in lp:~siab/+git/siab-dependencies that can be used for testing.
CI integration has not been considered yet - a setup-jenkaas would need to be added to the Makefile and would need to pull siab-dependencies
snapstore-library review here https:/
Unmerged commits
- 9038b0a... by Tony Simpson
-
Removed dead code
- e980150... by Tony Simpson
-
Make build process more like snap-store-proxy
Borrow approach from snapstore-snap (snap-store-proxy).
snap/plugins/* is a copy of plugin from snap-store-proxy. - 54341a0... by Tony Simpson
-
Integrate snapstore_library auth and overrides
* Test have been rewritten, still requires a few more I think
* Dead code has not been removed yet
* Has not been user tested
Preview Diff
1 | diff --git a/Makefile b/Makefile |
2 | index d8f1bd4..2266b98 100644 |
3 | --- a/Makefile |
4 | +++ b/Makefile |
5 | @@ -1,15 +1,22 @@ |
6 | -# In a global tmp dir for now so snapcraft doesn't make copies (see |
7 | -# https://forum.snapcraft.io/t/customizable-ignores/230/12). |
8 | -TMPDIR = /tmp/snapstore-client.tmp |
9 | - |
10 | +FILE_DEPS = Makefile snap/snapcraft.yaml snapstore snapstore_client/**/*.py dependencies/*.* |
11 | SERVICE_PACKAGE = snapstore_client |
12 | -ENV = $(TMPDIR)/env |
13 | +ENV = env |
14 | PYTHON3 = $(ENV)/bin/python3 |
15 | PIP = $(PYTHON3) -m pip |
16 | FLAKE8 = $(ENV)/bin/flake8 |
17 | |
18 | DEPENDENCY_REPO ?= lp:~siab/+git/siab-dependencies |
19 | -SNAPSTORE_DEPENDENCY_DIR ?= $(TMPDIR)/dependencies |
20 | +SNAPSTORE_DEPENDENCY_DIR = dependencies |
21 | + |
22 | +SNAPCRAFT ?= snapcraft |
23 | +VIRT := $(shell systemd-detect-virt) |
24 | +ifeq ($(VIRT),lxc) |
25 | + SNAPCRAFT_FLAGS ?= --destructive-mode |
26 | +else |
27 | + SNAPCRAFT_FLAGS ?= --use-lxd |
28 | +endif |
29 | +SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT ?= edge |
30 | +export PATH := $(PATH):/snap/bin |
31 | |
32 | |
33 | $(SNAPSTORE_DEPENDENCY_DIR): |
34 | @@ -26,8 +33,13 @@ $(ENV)/dev: $(ENV)/prod |
35 | |
36 | bootstrap: $(ENV)/prod |
37 | |
38 | -snap: |
39 | - snapcraft cleanbuild |
40 | +snap-store-proxy-client.snap: $(SNAPSTORE_DEPENDENCY_DIR) $(FILE_DEPS) |
41 | + SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT=${SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT} SNAPCRAFT_ENABLE_ERROR_REPORTING=0 $(SNAPCRAFT) snap $(SNAPCRAFT_FLAGS) --output snap-store-proxy-client.snap |
42 | + |
43 | +snap: snap-store-proxy-client.snap |
44 | + |
45 | +install: snap |
46 | + sudo snap install snap-store-proxy-client.snap --dangerous |
47 | |
48 | test: $(ENV)/dev |
49 | $(PYTHON3) -m unittest $(TESTS) 2>&1 |
50 | @@ -49,11 +61,19 @@ coverage: $(ENV)/dev |
51 | $(PYTHON3) -m coverage html |
52 | |
53 | clean: |
54 | - rm -rf $(TMPDIR) |
55 | +ifneq (,$(findstring --destructive-mode, $(SNAPCRAFT_FLAGS))) |
56 | + rm -fr parts |
57 | + rm -fr stage |
58 | +else |
59 | + $(SNAPCRAFT) clean $(SNAPCRAFT_FLAGS) |
60 | +endif |
61 | + rm -rf env |
62 | rm -rf dist docs/build |
63 | rm -rf .coverage htmlcov |
64 | - find -name '__pycache__' -print0 | xargs -0 rm -rf |
65 | - find -name '*.~*' -delete |
66 | + snapcraft clean |
67 | + |
68 | +fullclean: clean |
69 | + rm -rf dependencies |
70 | |
71 | |
72 | -.PHONY: bootstrap test lint coverage clean snap docs |
73 | +.PHONY: bootstrap test lint coverage clean fullclean snap install docs |
74 | diff --git a/requirements.txt b/requirements.txt |
75 | index 09ae15a..19a551f 100644 |
76 | --- a/requirements.txt |
77 | +++ b/requirements.txt |
78 | @@ -2,3 +2,4 @@ pymacaroons |
79 | pysha3 |
80 | pyxdg |
81 | requests |
82 | +snapstore_library |
83 | \ No newline at end of file |
84 | diff --git a/snap/plugins/_python/__init__.py b/snap/plugins/_python/__init__.py |
85 | new file mode 100644 |
86 | index 0000000..6d6ad33 |
87 | --- /dev/null |
88 | +++ b/snap/plugins/_python/__init__.py |
89 | @@ -0,0 +1,19 @@ |
90 | +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- |
91 | +# |
92 | +# Copyright (C) 2017 Canonical Ltd |
93 | +# |
94 | +# This program is free software: you can redistribute it and/or modify |
95 | +# it under the terms of the GNU General Public License version 3 as |
96 | +# published by the Free Software Foundation. |
97 | +# |
98 | +# This program is distributed in the hope that it will be useful, |
99 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
100 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
101 | +# GNU General Public License for more details. |
102 | +# |
103 | +# You should have received a copy of the GNU General Public License |
104 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
105 | + |
106 | +from ._pip import Pip # noqa |
107 | +from ._python_finder import get_python_command # noqa |
108 | +from ._sitecustomize import generate_sitecustomize # noqa |
109 | diff --git a/snap/plugins/_python/_pip.py b/snap/plugins/_python/_pip.py |
110 | new file mode 100644 |
111 | index 0000000..9acf3ba |
112 | --- /dev/null |
113 | +++ b/snap/plugins/_python/_pip.py |
114 | @@ -0,0 +1,575 @@ |
115 | +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- |
116 | +# |
117 | +# Copyright (C) 2016-2019 Canonical Ltd |
118 | +# |
119 | +# This program is free software: you can redistribute it and/or modify |
120 | +# it under the terms of the GNU General Public License version 3 as |
121 | +# published by the Free Software Foundation. |
122 | +# |
123 | +# This program is distributed in the hope that it will be useful, |
124 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
125 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
126 | +# GNU General Public License for more details. |
127 | +# |
128 | +# You should have received a copy of the GNU General Public License |
129 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
130 | + |
131 | +import collections |
132 | +import contextlib |
133 | +import json |
134 | +import logging |
135 | +import os |
136 | +import re |
137 | +import shutil |
138 | +import stat |
139 | +import subprocess |
140 | +import sys |
141 | +import tempfile |
142 | +from typing import List, Optional, Sequence, Set |
143 | + |
144 | +import snapcraft |
145 | +from snapcraft import file_utils |
146 | +from snapcraft.internal import mangling |
147 | +from ._python_finder import get_python_command, get_python_headers, get_python_home |
148 | +from . import errors |
149 | + |
150 | +logger = logging.getLogger(__name__) |
151 | + |
152 | + |
153 | +def _process_common_args( |
154 | + *, constraints: Optional[Set[str]] = None, process_dependency_links: bool = False |
155 | +) -> List[str]: |
156 | + args = [] |
157 | + if constraints: |
158 | + for constraint in constraints: |
159 | + args.extend(["--constraint", constraint]) |
160 | + |
161 | + if process_dependency_links: |
162 | + args.append("--process-dependency-links") |
163 | + |
164 | + return args |
165 | + |
166 | + |
167 | +def _process_package_args( |
168 | + *, packages: Sequence[str], requirements: Sequence[str], setup_py_dir: str |
169 | +) -> List[str]: |
170 | + args = [] |
171 | + if requirements: |
172 | + for requirement in requirements: |
173 | + args.extend(["--requirement", requirement]) |
174 | + |
175 | + if packages: |
176 | + args.extend(packages) |
177 | + |
178 | + if setup_py_dir: |
179 | + args.append(".") |
180 | + |
181 | + return args |
182 | + |
183 | + |
184 | +def _replicate_owner_mode(path): |
185 | + # Don't bother with a path that doesn't exist or is a symlink. The target |
186 | + # of the symlink will either be updated anyway, or we won't have permission |
187 | + # to change it. |
188 | + if not os.path.exists(path) or os.path.islink(path): |
189 | + return |
190 | + |
191 | + file_mode = os.stat(path).st_mode |
192 | + |
193 | + # We at least need to write to it to fix shebangs later |
194 | + new_mode = file_mode | stat.S_IWUSR |
195 | + |
196 | + # If the owner can execute it, so should everyone else. |
197 | + if file_mode & stat.S_IXUSR: |
198 | + new_mode |= stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH |
199 | + |
200 | + # If the owner can read it, so should everyone else |
201 | + if file_mode & stat.S_IRUSR: |
202 | + new_mode |= stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH |
203 | + |
204 | + os.chmod(path, new_mode) |
205 | + |
206 | + |
207 | +def _fix_permissions(path): |
208 | + for root, dirs, files in os.walk(path): |
209 | + for filename in files: |
210 | + _replicate_owner_mode(os.path.join(root, filename)) |
211 | + for dirname in dirs: |
212 | + _replicate_owner_mode(os.path.join(root, dirname)) |
213 | + |
214 | + |
215 | +def _expand_vars(text, project_dir): |
216 | + text = text.replace('$SNAPCRAFT_PROJECT_DIR', project_dir) |
217 | + text = text.replace('${SNAPCRAFT_PROJECT_DIR}', project_dir) |
218 | + return text |
219 | + |
220 | + |
221 | +class Pip: |
222 | + """Wrapper for pip abstracting the args necessary for use in a part. |
223 | + |
224 | + This class takes care of fetching pip, setuptools, and wheel, and then |
225 | + simply shells out to pip with the magical arguments necessary to install |
226 | + packages into a part. |
227 | + |
228 | + Of particular importance: packages must be downloaded (via download()) |
229 | + before they can be installed or have wheels built. |
230 | + """ |
231 | + |
232 | + def __init__(self, *, python_major_version, part_dir, install_dir, stage_dir, project_dir): |
233 | + """Initialize pip. |
234 | + |
235 | + You must call setup() before you can actually use pip. |
236 | + |
237 | + :param str python_major_version: The python major version to find (2 or |
238 | + 3) |
239 | + :param str part_dir: Path to the part's working area |
240 | + :param str install_dir: Path to the part's install area |
241 | + :param str stage_dir: Path to the staging area |
242 | + :param str project_dir: Path to the project area |
243 | + |
244 | + :raises MissingPythonCommandError: If no python could be found in the |
245 | + staging or part's install area. |
246 | + """ |
247 | + self._python_major_version = python_major_version |
248 | + self._install_dir = install_dir |
249 | + self._stage_dir = stage_dir |
250 | + self._project_dir = project_dir |
251 | + |
252 | + self._python_package_dir = os.path.join(part_dir, "python-packages") |
253 | + os.makedirs(self._python_package_dir, exist_ok=True) |
254 | + |
255 | + self.__python_command = None # type:str |
256 | + self.__python_home = None # type: str |
257 | + |
258 | + @property |
259 | + def _python_command(self): |
260 | + """Lazily determine the python command required.""" |
261 | + if not self.__python_command: |
262 | + self.__python_command = get_python_command( |
263 | + self._python_major_version, |
264 | + stage_dir=self._stage_dir, |
265 | + install_dir=self._install_dir, |
266 | + ) |
267 | + return self.__python_command |
268 | + |
269 | + @property |
270 | + def _python_home(self): |
271 | + """Lazily determine the correct python home.""" |
272 | + if not self.__python_home: |
273 | + self.__python_home = get_python_home( |
274 | + self._python_major_version, |
275 | + stage_dir=self._stage_dir, |
276 | + install_dir=self._install_dir, |
277 | + ) |
278 | + return self.__python_home |
279 | + |
280 | + def setup( |
281 | + self, |
282 | + no_index: bool = False, |
283 | + find_links: Optional[Sequence[str]] = None |
284 | + ): |
285 | + """Install pip and dependencies. |
286 | + |
287 | + Check to see if pip has already been installed. If not, fetch pip, |
288 | + setuptools, and wheel, and install them so they can be used. |
289 | + """ |
290 | + |
291 | + self._ensure_pip_installed( |
292 | + no_index=no_index, |
293 | + find_links=find_links |
294 | + ) |
295 | + self._ensure_wheel_installed( |
296 | + no_index=no_index, |
297 | + find_links=find_links |
298 | + ) |
299 | + self._ensure_setuptools_installed( |
300 | + no_index=no_index, |
301 | + find_links=find_links |
302 | + ) |
303 | + |
304 | + def is_setup(self): |
305 | + """Return true if this class has already been setup.""" |
306 | + |
307 | + return ( |
308 | + self._is_pip_installed() |
309 | + and self._is_wheel_installed() |
310 | + and self._is_setuptools_installed() |
311 | + ) |
312 | + |
313 | + def _ensure_pip_installed( |
314 | + self, |
315 | + no_index: bool = False, |
316 | + find_links: Optional[Sequence[str]] = None |
317 | + ): |
318 | + # Check to see if we have our own pip. If not, we need to use the pip |
319 | + # on the host (installed via build-packages) to grab our own. |
320 | + if not self._is_pip_installed(): |
321 | + logger.info("Fetching and installing pip...") |
322 | + |
323 | + real_python_home = self.__python_home |
324 | + |
325 | + # Make sure we're using pip from the host. Wrapping this operation |
326 | + # in a try/finally to make sure we revert away from the host's |
327 | + # python at the end. |
328 | + try: |
329 | + self.__python_home = os.path.join(os.path.sep, "usr") |
330 | + |
331 | + # Using the host's pip, install our own pip |
332 | + self.download( |
333 | + {"pip"}, |
334 | + no_index=no_index, |
335 | + find_links=find_links |
336 | + ) |
337 | + self.install({"pip"}, ignore_installed=True) |
338 | + finally: |
339 | + # Now that we have our own pip, reset the python home |
340 | + self.__python_home = real_python_home |
341 | + |
342 | + def _ensure_wheel_installed( |
343 | + self, |
344 | + no_index: bool = False, |
345 | + find_links: Optional[Sequence[str]] = None |
346 | + ): |
347 | + if not self._is_wheel_installed(): |
348 | + logger.info("Fetching and installing wheel...") |
349 | + self.download( |
350 | + {"wheel"}, |
351 | + no_index=no_index, |
352 | + find_links=find_links |
353 | + ) |
354 | + self.install({"wheel"}, ignore_installed=True) |
355 | + |
356 | + def _ensure_setuptools_installed( |
357 | + self, |
358 | + no_index: bool = False, |
359 | + find_links: Optional[Sequence[str]] = None |
360 | + ): |
361 | + if not self._is_setuptools_installed(): |
362 | + logger.info("Fetching and installing setuptools...") |
363 | + self.download( |
364 | + {"setuptools"}, |
365 | + no_index=no_index, |
366 | + find_links=find_links |
367 | + ) |
368 | + self.install({"setuptools"}, ignore_installed=True) |
369 | + |
370 | + def _is_pip_installed(self): |
371 | + try: |
372 | + # We're expecting an error here at least once complaining about |
373 | + # pip not being installed. In order to verify that the error is the |
374 | + # one we think it is, we need to process the stderr. So we'll |
375 | + # redirect it to stdout. If it's not the error we expect, something |
376 | + # is wrong, so re-raise it. |
377 | + # |
378 | + # Using _run_output here so stdout doesn't get printed to the |
379 | + # terminal. |
380 | + self._run_output([], stderr=subprocess.STDOUT) |
381 | + except subprocess.CalledProcessError as e: |
382 | + output = e.output.decode(sys.getfilesystemencoding()).strip() |
383 | + if "no module named pip" in output.lower(): |
384 | + return False |
385 | + else: |
386 | + raise e |
387 | + return True |
388 | + |
389 | + def _is_wheel_installed(self): |
390 | + return "wheel" in self.list() |
391 | + |
392 | + def _is_setuptools_installed(self): |
393 | + return "setuptools" in self.list() |
394 | + |
395 | + def download( |
396 | + self, |
397 | + packages, |
398 | + *, |
399 | + setup_py_dir: Optional[str] = None, |
400 | + constraints: Optional[Set[str]] = None, |
401 | + requirements: Optional[Sequence[str]] = None, |
402 | + process_dependency_links: bool = False, |
403 | + no_index: bool = False, |
404 | + find_links: Optional[Sequence[str]] = None |
405 | + ): |
406 | + """Download packages into cache, but do not install them. |
407 | + |
408 | + :param iterable packages: Packages to download from index. |
409 | + :param str setup_py_dir: Directory containing setup.py. |
410 | + :param iterable constraints: Collection of paths to constraints files. |
411 | + :param iterable requirements: Collection of paths to requirements |
412 | + files. |
413 | + :param boolean process_dependency_links: Enable the processing of |
414 | + dependency links. |
415 | + :param boolean no_index: Don't hit PyPI |
416 | + :param iterable find_links: Locations of additional repositories |
417 | + """ |
418 | + package_args = _process_package_args( |
419 | + packages=packages, requirements=requirements, setup_py_dir=setup_py_dir |
420 | + ) |
421 | + |
422 | + if not package_args: |
423 | + return # No operation was requested |
424 | + |
425 | + args = _process_common_args( |
426 | + process_dependency_links=process_dependency_links, constraints=constraints |
427 | + ) |
428 | + if no_index: |
429 | + args.append('--no-index') |
430 | + if find_links: |
431 | + for fl_item in find_links: |
432 | + args.append('--find-links') |
433 | + args.append(_expand_vars(fl_item, self._project_dir)) |
434 | + |
435 | + # Using pip with a few special parameters: |
436 | + # |
437 | + # --disable-pip-version-check: Don't whine if pip is out-of-date with |
438 | + # the version on pypi. |
439 | + # --dest: Download packages into the directory we've set aside for it. |
440 | + # |
441 | + # For cwd, setup_py_dir will be the actual directory we need to be in |
442 | + # or None. |
443 | + self._run( |
444 | + [ |
445 | + "download", |
446 | + "--disable-pip-version-check", |
447 | + "--dest", |
448 | + self._python_package_dir, |
449 | + ] |
450 | + + args |
451 | + + package_args, |
452 | + cwd=setup_py_dir, |
453 | + ) |
454 | + |
455 | + def install( |
456 | + self, |
457 | + packages, |
458 | + *, |
459 | + setup_py_dir: Optional[str] = None, |
460 | + constraints: Optional[Set[str]] = None, |
461 | + requirements: Optional[Sequence[str]] = None, |
462 | + process_dependency_links: bool = False, |
463 | + upgrade: bool = False, |
464 | + install_deps: bool = True, |
465 | + ignore_installed: bool = False |
466 | + ): |
467 | + """Install packages from cache. |
468 | + |
469 | + The packages should have already been downloaded via `download()`. |
470 | + |
471 | + :param iterable packages: Packages to install from cache. |
472 | + :param str setup_py_dir: Directory containing setup.py. |
473 | + :param iterable constraints: Collection of paths to constraints files. |
474 | + :param iterable requirements: Collection of paths to requirements |
475 | + files. |
476 | + :param boolean process_dependency_links: Enable the processing of |
477 | + dependency links. |
478 | + :param boolean upgrade: Recursively upgrade packages. |
479 | + :param boolean install_deps: Install package dependencies. |
480 | + :param boolean ignore_installed: Reinstall packages if they're already |
481 | + installed |
482 | + """ |
483 | + package_args = _process_package_args( |
484 | + packages=packages, requirements=requirements, setup_py_dir=setup_py_dir |
485 | + ) |
486 | + |
487 | + if not package_args: |
488 | + return # No operation was requested |
489 | + |
490 | + args = _process_common_args( |
491 | + process_dependency_links=process_dependency_links, constraints=constraints |
492 | + ) |
493 | + |
494 | + if upgrade: |
495 | + args.append("--upgrade") |
496 | + |
497 | + if not install_deps: |
498 | + args.append("--no-deps") |
499 | + |
500 | + if ignore_installed: |
501 | + args.append("--ignore-installed") |
502 | + |
503 | + # Using pip with a few special parameters: |
504 | + # |
505 | + # --user: Install packages to PYTHONUSERBASE, which we've pointed to |
506 | + # the installdir. |
507 | + # --no-index: Don't hit pypi, assume the packages are already |
508 | + # downloaded (i.e. by using `self.download()`) |
509 | + # --find-links: Provide the directory into which the packages should |
510 | + # have already been fetched |
511 | + # |
512 | + # For cwd, setup_py_dir will be the actual directory we need to be in |
513 | + # or None. |
514 | + self._run( |
515 | + [ |
516 | + "install", |
517 | + "--user", |
518 | + "--no-index", |
519 | + "--find-links", |
520 | + self._python_package_dir, |
521 | + ] |
522 | + + args |
523 | + + package_args, |
524 | + cwd=setup_py_dir, |
525 | + ) |
526 | + |
527 | + # Installing with --user results in a directory with 700 permissions. |
528 | + # We need it a bit more open than that, so open it up. |
529 | + _fix_permissions(self._install_dir) |
530 | + |
531 | + # Fix all shebangs to use the in-snap python. |
532 | + mangling.rewrite_python_shebangs(self._install_dir) |
533 | + |
534 | + def wheel( |
535 | + self, |
536 | + packages, |
537 | + *, |
538 | + setup_py_dir: Optional[str] = None, |
539 | + constraints: Optional[Set[str]] = None, |
540 | + requirements: Optional[Sequence[str]] = None, |
541 | + process_dependency_links: bool = False |
542 | + ): |
543 | + """Build wheels of packages in the cache. |
544 | + |
545 | + The packages should have already been downloaded via `download()`. |
546 | + |
547 | + :param iterable packages: Packages in cache for which to build wheels. |
548 | + :param str setup_py_dir: Directory containing setup.py. |
549 | + :param iterable constraints: Collection of paths to constraints files. |
550 | + :param iterable requirements: Collection of paths to requirements |
551 | + files. |
552 | + :param boolean process_dependency_links: Enable the processing of |
553 | + dependency links. |
554 | + |
555 | + :return: List of paths to each wheel that was built. |
556 | + :rtype: list |
557 | + """ |
558 | + package_args = _process_package_args( |
559 | + packages=packages, requirements=requirements, setup_py_dir=setup_py_dir |
560 | + ) |
561 | + |
562 | + if not package_args: |
563 | + return [] # No operation was requested |
564 | + |
565 | + args = _process_common_args( |
566 | + process_dependency_links=process_dependency_links, constraints=constraints |
567 | + ) |
568 | + |
569 | + wheels = [] # type: List[str] |
570 | + with tempfile.TemporaryDirectory() as temp_dir: |
571 | + |
572 | + # Using pip with a few special parameters: |
573 | + # |
574 | + # --no-index: Don't hit pypi, assume the packages are already |
575 | + # downloaded (i.e. by using `self.download()`) |
576 | + # --find-links: Provide the directory into which the packages |
577 | + # should have already been fetched |
578 | + # --wheel-dir: Build wheels into a temporary working area rather |
579 | + # rather than cwd. We'll copy them over. FIXME: We can |
580 | + # probably get away just building them in the package |
581 | + # dir. Try that once this refactor has been validated. |
582 | + self._run( |
583 | + [ |
584 | + "wheel", |
585 | + "--no-index", |
586 | + "--find-links", |
587 | + self._python_package_dir, |
588 | + "--wheel-dir", |
589 | + temp_dir, |
590 | + ] |
591 | + + args |
592 | + + package_args, |
593 | + cwd=setup_py_dir, |
594 | + ) |
595 | + wheels = os.listdir(temp_dir) |
596 | + for wheel in wheels: |
597 | + file_utils.link_or_copy( |
598 | + os.path.join(temp_dir, wheel), |
599 | + os.path.join(self._python_package_dir, wheel), |
600 | + ) |
601 | + |
602 | + return [os.path.join(self._python_package_dir, wheel) for wheel in wheels] |
603 | + |
604 | + def list(self, *, user=False): |
605 | + """Determine which packages have been installed. |
606 | + |
607 | + :param boolean user: Whether or not to limit results to user base. |
608 | + |
609 | + :return: Dict of installed python packages and their versions |
610 | + :rtype: dict |
611 | + """ |
612 | + command = ["list"] |
613 | + if user: |
614 | + command.append("--user") |
615 | + |
616 | + packages = collections.OrderedDict() |
617 | + try: |
618 | + output = self._run_output(command + ["--format=json"]) |
619 | + json_output = json.loads(output, object_pairs_hook=collections.OrderedDict) |
620 | + except subprocess.CalledProcessError: |
621 | + # --format requires a newer pip, so fall back to legacy output |
622 | + output = self._run_output(command) |
623 | + json_output = [] # type: List[Dict[str, str]] |
624 | + version_regex = re.compile(r"\((.+)\)") |
625 | + for line in output.splitlines(): |
626 | + line = line.split() |
627 | + m = version_regex.search(line[1]) |
628 | + if not m: |
629 | + raise errors.PipListInvalidLegacyFormatError(output) |
630 | + json_output.append({"name": line[0], "version": m.group(1)}) |
631 | + except json.decoder.JSONDecodeError as e: |
632 | + raise errors.PipListInvalidJsonError(output) from e |
633 | + |
634 | + for package in json_output: |
635 | + if "name" not in package: |
636 | + raise errors.PipListMissingFieldError("name", output) |
637 | + if "version" not in package: |
638 | + raise errors.PipListMissingFieldError("version", output) |
639 | + packages[package["name"]] = package["version"] |
640 | + return packages |
641 | + |
642 | + def clean_packages(self): |
643 | + """Remove the package cache.""" |
644 | + with contextlib.suppress(FileNotFoundError): |
645 | + shutil.rmtree(self._python_package_dir) |
646 | + |
647 | + def env(self): |
648 | + """The environment used by pip. |
649 | + |
650 | + This function is only useful if you happen to need to call into pip's |
651 | + environment without using the API otherwise made available here (e.g. |
652 | + calling the setup.py directly instead of with pip). |
653 | + |
654 | + :return: Dict of the environment necessary to use the pip contained |
655 | + here. |
656 | + :rtype: dict |
657 | + """ |
658 | + env = os.environ.copy() |
659 | + env["PYTHONUSERBASE"] = self._install_dir |
660 | + env["PYTHONHOME"] = self._python_home |
661 | + |
662 | + env["PATH"] = "{}:{}".format( |
663 | + os.path.join(self._install_dir, "usr", "bin"), os.path.expandvars("$PATH") |
664 | + ) |
665 | + |
666 | + headers = get_python_headers( |
667 | + self._python_major_version, stage_dir=self._stage_dir |
668 | + ) |
669 | + if headers: |
670 | + current_cppflags = env.get("CPPFLAGS", "") |
671 | + env["CPPFLAGS"] = "-I{}".format(headers) |
672 | + if current_cppflags: |
673 | + env["CPPFLAGS"] = "{} {}".format(env["CPPFLAGS"], current_cppflags) |
674 | + |
675 | + return env |
676 | + |
677 | + def _run(self, args, runner=None, **kwargs): |
678 | + env = self.env() |
679 | + |
680 | + # Using None as the default value instead of common.run so we can mock |
681 | + # common.run. |
682 | + if runner is None: |
683 | + runner = snapcraft.internal.common.run |
684 | + return runner( |
685 | + [self._python_command, "-m", "pip"] + list(args), env=env, **kwargs |
686 | + ) |
687 | + |
688 | + def _run_output(self, args, **kwargs): |
689 | + return self._run(args, runner=snapcraft.internal.common.run_output, **kwargs) |
690 | diff --git a/snap/plugins/_python/_python_finder.py b/snap/plugins/_python/_python_finder.py |
691 | new file mode 100644 |
692 | index 0000000..c677cd6 |
693 | --- /dev/null |
694 | +++ b/snap/plugins/_python/_python_finder.py |
695 | @@ -0,0 +1,104 @@ |
696 | +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- |
697 | +# |
698 | +# Copyright (C) 2017 Canonical Ltd |
699 | +# |
700 | +# This program is free software: you can redistribute it and/or modify |
701 | +# it under the terms of the GNU General Public License version 3 as |
702 | +# published by the Free Software Foundation. |
703 | +# |
704 | +# This program is distributed in the hope that it will be useful, |
705 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
706 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
707 | +# GNU General Public License for more details. |
708 | +# |
709 | +# You should have received a copy of the GNU General Public License |
710 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
711 | + |
712 | +import os |
713 | +import glob |
714 | + |
715 | +from . import errors |
716 | + |
717 | + |
718 | +def get_python_command(python_major_version, *, stage_dir, install_dir): |
719 | + """Find the python command to use, preferring staged over the part. |
720 | + |
721 | + We prefer the staged python as opposed to the in-part python in order to |
722 | + support one part that supplies python, with another part built `after` it |
723 | + wanting to use its python. |
724 | + |
725 | + :param str python_major_version: The python major version to find (2 or 3) |
726 | + :param str stage_dir: Path to the staging area |
727 | + :param str install_dir: Path to the part's install area |
728 | + |
729 | + :return: Path to the python command that was found |
730 | + :rtype: str |
731 | + |
732 | + :raises MissingPythonCommandError: If no python could be found in the |
733 | + staging or part's install area. |
734 | + """ |
735 | + python_command_name = "python{}".format(python_major_version) |
736 | + python_command = os.path.join("usr", "bin", python_command_name) |
737 | + staged_python = os.path.join(stage_dir, python_command) |
738 | + part_python = os.path.join(install_dir, python_command) |
739 | + |
740 | + if os.path.exists(staged_python): |
741 | + return staged_python |
742 | + elif os.path.exists(part_python): |
743 | + return part_python |
744 | + else: |
745 | + raise errors.MissingPythonCommandError( |
746 | + python_command_name, [stage_dir, install_dir] |
747 | + ) |
748 | + |
749 | + |
750 | +def get_python_headers(python_major_version, *, stage_dir): |
751 | + """Find the python headers to use, if any, preferring staged over the host. |
752 | + |
753 | + We want to make sure we use the headers from the staging area if available, |
754 | + or we may end up building for an older python version than the one we |
755 | + actually want to use. |
756 | + |
757 | + :param str python_major_version: The python version to find (2 or 3) |
758 | + :param str stage_dir: Path to the staging area |
759 | + |
760 | + :return: Path to the python headers that were found ('' if none) |
761 | + :rtype: str |
762 | + """ |
763 | + python_command_name = "python{}".format(python_major_version) |
764 | + base_match = os.path.join("usr", "include", "{}*".format(python_command_name)) |
765 | + staged_python = glob.glob(os.path.join(stage_dir, base_match)) |
766 | + host_python = glob.glob(os.path.join(os.path.sep, base_match)) |
767 | + |
768 | + if staged_python: |
769 | + return staged_python[0] |
770 | + elif host_python: |
771 | + return host_python[0] |
772 | + else: |
773 | + return "" |
774 | + |
775 | + |
776 | +def get_python_home(python_major_version, *, stage_dir, install_dir): |
777 | + """Find the correct PYTHONHOME, preferring staged over the part. |
778 | + |
779 | + We prefer the staged python as opposed to the in-part python in order to |
780 | + support one part that supplies python, with another part built `after` it |
781 | + wanting to use its python. |
782 | + |
783 | + :param str python_major_version: The python major version to find (2 or 3) |
784 | + :param str stage_dir: Path to the staging area |
785 | + :param str install_dir: Path to the part's install area |
786 | + |
787 | + :return: Path to the PYTHONHOME that was found |
788 | + :rtype: str |
789 | + |
790 | + :raises MissingPythonCommandError: If no python could be found in the |
791 | + staging or part's install area. |
792 | + """ |
793 | + python_command = get_python_command( |
794 | + python_major_version, stage_dir=stage_dir, install_dir=install_dir |
795 | + ) |
796 | + if python_command.startswith(stage_dir): |
797 | + return os.path.join(stage_dir, "usr") |
798 | + else: |
799 | + return os.path.join(install_dir, "usr") |
800 | diff --git a/snap/plugins/_python/_sitecustomize.py b/snap/plugins/_python/_sitecustomize.py |
801 | new file mode 100644 |
802 | index 0000000..aed89ea |
803 | --- /dev/null |
804 | +++ b/snap/plugins/_python/_sitecustomize.py |
805 | @@ -0,0 +1,124 @@ |
806 | +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- |
807 | +# |
808 | +# Copyright (C) 2017 Canonical Ltd |
809 | +# |
810 | +# This program is free software: you can redistribute it and/or modify |
811 | +# it under the terms of the GNU General Public License version 3 as |
812 | +# published by the Free Software Foundation. |
813 | +# |
814 | +# This program is distributed in the hope that it will be useful, |
815 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
816 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
817 | +# GNU General Public License for more details. |
818 | +# |
819 | +# You should have received a copy of the GNU General Public License |
820 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
821 | + |
822 | +import contextlib |
823 | +import glob |
824 | +import os |
825 | +from textwrap import dedent |
826 | + |
827 | +from ._python_finder import get_python_command |
828 | +from . import errors |
829 | + |
830 | +_SITECUSTOMIZE_TEMPLATE = dedent( |
831 | + """\ |
832 | + import site |
833 | + import os |
834 | + |
835 | + snap_dir = os.getenv("SNAP") |
836 | + snapcraft_stage_dir = os.getenv("SNAPCRAFT_STAGE") |
837 | + snapcraft_part_install = os.getenv("SNAPCRAFT_PART_INSTALL") |
838 | + |
839 | + for d in (snap_dir, snapcraft_stage_dir, snapcraft_part_install): |
840 | + if d: |
841 | + site_dir = os.path.join(d, "{site_dir}") |
842 | + site.addsitedir(site_dir) |
843 | + |
844 | + if snap_dir: |
845 | + site.ENABLE_USER_SITE = False""" |
846 | +) |
847 | + |
848 | + |
849 | +def _get_user_site_dir(python_major_version, *, install_dir): |
850 | + path_glob = os.path.join( |
851 | + install_dir, "lib", "python{}*".format(python_major_version), "site-packages" |
852 | + ) |
853 | + user_site_dirs = glob.glob(path_glob) |
854 | + if not user_site_dirs: |
855 | + raise errors.MissingUserSitePackagesError(path_glob) |
856 | + |
857 | + return user_site_dirs[0][len(install_dir) + 1 :] |
858 | + |
859 | + |
860 | +def _get_sitecustomize_path(python_major_version, *, stage_dir, install_dir): |
861 | + # Use the part's install_dir unless there's a python in the staging area |
862 | + base_dir = install_dir |
863 | + with contextlib.suppress(errors.MissingPythonCommandError): |
864 | + python_command = get_python_command( |
865 | + python_major_version, stage_dir=stage_dir, install_dir=install_dir |
866 | + ) |
867 | + if python_command.startswith(stage_dir): |
868 | + base_dir = stage_dir |
869 | + |
870 | + site_py_glob = os.path.join( |
871 | + base_dir, "usr", "lib", "python{}*".format(python_major_version), "site.py" |
872 | + ) |
873 | + python_sites = glob.glob(site_py_glob) |
874 | + if not python_sites: |
875 | + raise errors.MissingSitePyError(site_py_glob) |
876 | + |
877 | + python_site_dir = os.path.dirname(python_sites[0]) |
878 | + |
879 | + return os.path.join( |
880 | + install_dir, python_site_dir[len(base_dir) + 1 :], "sitecustomize.py" |
881 | + ) |
882 | + |
883 | + |
884 | +def generate_sitecustomize(python_major_version, *, stage_dir, install_dir): |
885 | + """Generate a sitecustomize.py to look in staging, part install, and snap. |
886 | + |
887 | + This is done by checking the values of the environment variables $SNAP, |
888 | + $SNAPCRAFT_STAGE, and $SNAPCRAFT_PART_INSTALL. As a result, the same |
889 | + sitecustomize.py works to find packages at both build- and run-time. |
890 | + |
891 | + :param str python_major_version: The python major version to use (2 or 3) |
892 | + :param str stage_dir: Path to the staging area |
893 | + :param str install_dir: Path to the part's install area |
894 | + |
895 | + :raises MissingUserSitePackagesError: If no user site packages are found in |
896 | + install_dir/lib/pythonX* |
897 | + :raises MissingSitePyError: If no site.py can be found in either the |
898 | + staging area or the part install area. |
899 | + """ |
900 | + sitecustomize_path = _get_sitecustomize_path( |
901 | + python_major_version, stage_dir=stage_dir, install_dir=install_dir |
902 | + ) |
903 | + os.makedirs(os.path.dirname(sitecustomize_path), exist_ok=True) |
904 | + |
905 | + # There may very well already be a sitecustomize.py already there. If so, |
906 | + # get rid of it. Is may be a symlink to another sitecustomize.py, in which |
907 | + # case, we'll get rid of that one as well. |
908 | + if os.path.islink(sitecustomize_path): |
909 | + target_path = os.path.realpath(sitecustomize_path) |
910 | + |
911 | + # Only remove the target if it's contained within the install directory |
912 | + if target_path.startswith(os.path.abspath(install_dir) + os.sep): |
913 | + with contextlib.suppress(FileNotFoundError): |
914 | + os.remove(target_path) |
915 | + |
916 | + with contextlib.suppress(FileNotFoundError): |
917 | + os.remove(sitecustomize_path) |
918 | + |
919 | + # Create our sitecustomize. Python from the archives already has one |
920 | + # which is distro-specific and not needed here, so we truncate it if it's |
921 | + # already there. |
922 | + with open(sitecustomize_path, "w") as f: |
923 | + f.write( |
924 | + _SITECUSTOMIZE_TEMPLATE.format( |
925 | + site_dir=_get_user_site_dir( |
926 | + python_major_version, install_dir=install_dir |
927 | + ) |
928 | + ) |
929 | + ) |
930 | diff --git a/snap/plugins/_python/errors.py b/snap/plugins/_python/errors.py |
931 | new file mode 100644 |
932 | index 0000000..87fbcfb |
933 | --- /dev/null |
934 | +++ b/snap/plugins/_python/errors.py |
935 | @@ -0,0 +1,79 @@ |
936 | +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- |
937 | +# |
938 | +# Copyright (C) 2017 Canonical Ltd |
939 | +# |
940 | +# This program is free software: you can redistribute it and/or modify |
941 | +# it under the terms of the GNU General Public License version 3 as |
942 | +# published by the Free Software Foundation. |
943 | +# |
944 | +# This program is distributed in the hope that it will be useful, |
945 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
946 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
947 | +# GNU General Public License for more details. |
948 | +# |
949 | +# You should have received a copy of the GNU General Public License |
950 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
951 | + |
952 | +import snapcraft.internal.errors |
953 | +import snapcraft.formatting_utils |
954 | + |
955 | + |
956 | +class PythonPluginError(snapcraft.internal.errors.SnapcraftError): |
957 | + pass |
958 | + |
959 | + |
960 | +class MissingPythonCommandError(PythonPluginError): |
961 | + |
962 | + fmt = "Unable to find {python_version}, searched: {search_paths}" |
963 | + |
964 | + def __init__(self, python_version, search_paths): |
965 | + super().__init__( |
966 | + python_version=python_version, |
967 | + search_paths=snapcraft.formatting_utils.combine_paths( |
968 | + search_paths, "", ":" |
969 | + ), |
970 | + ) |
971 | + |
972 | + |
973 | +class MissingUserSitePackagesError(PythonPluginError): |
974 | + |
975 | + fmt = "Unable to find user site packages: {site_dir_glob}" |
976 | + |
977 | + def __init__(self, site_dir_glob): |
978 | + super().__init__(site_dir_glob=site_dir_glob) |
979 | + |
980 | + |
981 | +class MissingSitePyError(PythonPluginError): |
982 | + |
983 | + fmt = "Unable to find site.py: {site_py_glob}" |
984 | + |
985 | + def __init__(self, site_py_glob): |
986 | + super().__init__(site_py_glob=site_py_glob) |
987 | + |
988 | + |
989 | +class PipListInvalidLegacyFormatError(PythonPluginError): |
990 | + |
991 | + fmt = ( |
992 | + "Failed to parse Python package list: " |
993 | + "The returned output is not in the expected format:\n" |
994 | + "{output}" |
995 | + ) |
996 | + |
997 | + def __init__(self, output): |
998 | + super().__init__(output=output) |
999 | + |
1000 | + |
1001 | +class PipListInvalidJsonError(PythonPluginError): |
1002 | + |
1003 | + fmt = "Pip packages output isn't valid json: {json!r}" |
1004 | + |
1005 | + def __init__(self, json): |
1006 | + super().__init__(json=json) |
1007 | + |
1008 | + |
1009 | +class PipListMissingFieldError(PythonPluginError): |
1010 | + |
1011 | + fmt = "Pip packages json missing {field!r} field: {json!r}" |
1012 | + |
1013 | + def __init__(self, field, json): |
1014 | + super().__init__(field=field, json=json) |
1015 | diff --git a/snap/plugins/python_ols.py b/snap/plugins/python_ols.py |
1016 | new file mode 100644 |
1017 | index 0000000..95598d1 |
1018 | --- /dev/null |
1019 | +++ b/snap/plugins/python_ols.py |
1020 | @@ -0,0 +1,499 @@ |
1021 | +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- |
1022 | +# |
1023 | +# Copyright (C) 2016-2018 Canonical Ltd |
1024 | +# |
1025 | +# This program is free software: you can redistribute it and/or modify |
1026 | +# it under the terms of the GNU General Public License version 3 as |
1027 | +# published by the Free Software Foundation. |
1028 | +# |
1029 | +# This program is distributed in the hope that it will be useful, |
1030 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
1031 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
1032 | +# GNU General Public License for more details. |
1033 | +# |
1034 | +# You should have received a copy of the GNU General Public License |
1035 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
1036 | + |
1037 | +"""The python plugin can be used for python 2 or 3 based parts. |
1038 | + |
1039 | +It can be used for python projects where you would want to do: |
1040 | + |
1041 | + - import python modules with a requirements.txt |
1042 | + - build a python project that has a setup.py |
1043 | + - install packages straight from pip |
1044 | + |
1045 | +This plugin uses the common plugin keywords as well as those for "sources". |
1046 | +For more information check the 'plugins' topic for the former and the |
1047 | +'sources' topic for the latter. |
1048 | + |
1049 | +Additionally, this plugin uses the following plugin-specific keywords: |
1050 | + |
1051 | + - requirements: |
1052 | + (list of strings) |
1053 | + List of paths to requirements files. |
1054 | + - constraints: |
1055 | + (list of strings) |
1056 | + List of paths to constraint files. |
1057 | + - process-dependency-links: |
1058 | + (bool; default: false) |
1059 | + Enable the processing of dependency links in pip, which allow one |
1060 | + project to provide places to look for another project |
1061 | + - python-packages: |
1062 | + (list) |
1063 | + A list of dependencies to get from PyPI |
1064 | + - python-version: |
1065 | + (string; default: python3) |
1066 | + The python version to use. Valid options are: python2 and python3 |
1067 | + |
1068 | +If the plugin finds a python interpreter with a basename that matches |
1069 | +`python-version` in the <stage> directory on the following fixed path: |
1070 | +`<stage-dir>/usr/bin/<python-interpreter>` then this interpreter would |
1071 | +be preferred instead and no interpreter would be brought in through |
1072 | +`stage-packages` mechanisms. |
1073 | +""" |
1074 | + |
1075 | +import collections |
1076 | +import contextlib |
1077 | +import os |
1078 | +import re |
1079 | +from shutil import which |
1080 | +from textwrap import dedent |
1081 | +from typing import List, Set |
1082 | + |
1083 | +import requests |
1084 | + |
1085 | +import snapcraft |
1086 | +from snapcraft.common import isurl |
1087 | +from snapcraft.internal import errors, mangling |
1088 | +from snapcraft.internal.errors import SnapcraftPluginCommandError |
1089 | +import _python |
1090 | + |
1091 | + |
1092 | +class UnsupportedPythonVersionError(snapcraft.internal.errors.SnapcraftError): |
1093 | + |
1094 | + fmt = "Unsupported python version: {python_version!r}" |
1095 | + |
1096 | + |
1097 | +class SnapcraftPluginPythonFileMissing(snapcraft.internal.errors.SnapcraftError): |
1098 | + |
1099 | + fmt = ( |
1100 | + "Failed to find the referred {plugin_property} file at the given " |
1101 | + "path: {plugin_property_value!r}.\n" |
1102 | + "Check the property and ensure the file exists." |
1103 | + ) |
1104 | + |
1105 | + def __init__(self, *, plugin_property, plugin_property_value): |
1106 | + super().__init__( |
1107 | + plugin_property=plugin_property, plugin_property_value=plugin_property_value |
1108 | + ) |
1109 | + |
1110 | + |
1111 | +class PythonPlugin(snapcraft.BasePlugin): |
1112 | + @classmethod |
1113 | + def schema(cls): |
1114 | + schema = super().schema() |
1115 | + schema["properties"]["requirements"] = { |
1116 | + "type": "array", |
1117 | + "minitems": 1, |
1118 | + "uniqueItems": True, |
1119 | + "items": {"type": "string"}, |
1120 | + "default": [], |
1121 | + } |
1122 | + schema["properties"]["constraints"] = { |
1123 | + "type": "array", |
1124 | + "minitems": 1, |
1125 | + "uniqueItems": True, |
1126 | + "items": {"type": "string"}, |
1127 | + "default": [], |
1128 | + } |
1129 | + schema["properties"]["python-packages"] = { |
1130 | + "type": "array", |
1131 | + "minitems": 1, |
1132 | + "uniqueItems": True, |
1133 | + "items": {"type": "string"}, |
1134 | + "default": [], |
1135 | + } |
1136 | + schema["properties"]["process-dependency-links"] = { |
1137 | + "type": "boolean", |
1138 | + "default": False, |
1139 | + } |
1140 | + schema["properties"]["python-version"] = { |
1141 | + "type": "string", |
1142 | + "default": "python3", |
1143 | + "enum": ["python2", "python3"], |
1144 | + } |
1145 | + schema["properties"]["pip-no-index"] = { |
1146 | + "type": "boolean", |
1147 | + "default": False, |
1148 | + } |
1149 | + schema["properties"]["pip-find-links"] = { |
1150 | + "type": "array", |
1151 | + "items": {"type": "string"}, |
1152 | + "default": [], |
1153 | + } |
1154 | + schema["anyOf"] = [{"required": ["source"]}, {"required": ["python-packages"]}] |
1155 | + |
1156 | + return schema |
1157 | + |
1158 | + @classmethod |
1159 | + def get_pull_properties(cls): |
1160 | + # Inform Snapcraft of the properties associated with pulling. If these |
1161 | + # change in the YAML Snapcraft will consider the pull step dirty. |
1162 | + return [ |
1163 | + "requirements", |
1164 | + "constraints", |
1165 | + "python-packages", |
1166 | + "process-dependency-links", |
1167 | + "python-version", |
1168 | + "pip-no-index", |
1169 | + "pip-find-links", |
1170 | + ] |
1171 | + |
1172 | + @property |
1173 | + def plugin_stage_packages(self): |
1174 | + if self.options.python_version == "python2": |
1175 | + python_base = "python" |
1176 | + elif self.options.python_version == "python3": |
1177 | + python_base = "python3" |
1178 | + |
1179 | + if self.project.info.base in ("core", "core16", "core18"): |
1180 | + stage_packages = [python_base] |
1181 | + else: |
1182 | + stage_packages = [] |
1183 | + |
1184 | + if self.project.info.base == "core18" and python_base == "python3": |
1185 | + stage_packages.append("{}-distutils".format(python_base)) |
1186 | + |
1187 | + return stage_packages |
1188 | + |
1189 | + # ignore mypy error: Read-only property cannot override read-write property |
1190 | + @property # type: ignore |
1191 | + def stage_packages(self): |
1192 | + try: |
1193 | + _python.get_python_command( |
1194 | + self._python_major_version, |
1195 | + stage_dir=self.project.stage_dir, |
1196 | + install_dir=self.installdir, |
1197 | + ) |
1198 | + except _python.errors.MissingPythonCommandError: |
1199 | + return super().stage_packages + self.plugin_stage_packages |
1200 | + else: |
1201 | + return super().stage_packages |
1202 | + |
1203 | + @property |
1204 | + def _pip(self): |
1205 | + if not self.__pip: |
1206 | + self.__pip = _python.Pip( |
1207 | + python_major_version=self._python_major_version, |
1208 | + part_dir=self.partdir, |
1209 | + install_dir=self.installdir, |
1210 | + stage_dir=self.project.stage_dir, |
1211 | + project_dir=self.project._project_dir, |
1212 | + ) |
1213 | + return self.__pip |
1214 | + |
1215 | + def __init__(self, name, options, project): |
1216 | + super().__init__(name, options, project) |
1217 | + |
1218 | + self._setup_base_tools(project.info.base) |
1219 | + |
1220 | + self._manifest = collections.OrderedDict() |
1221 | + |
1222 | + # Pip requires only the major version of python rather than the command |
1223 | + # name like our option requires. |
1224 | + match = re.match(r"python(?P<major_version>\d).*", self.options.python_version) |
1225 | + if not match: |
1226 | + raise UnsupportedPythonVersionError( |
1227 | + python_version=self.options.python_version |
1228 | + ) |
1229 | + |
1230 | + self._python_major_version = match.group("major_version") |
1231 | + self.__pip = None |
1232 | + |
1233 | + def _setup_base_tools(self, base): |
1234 | + # NOTE: stage-packages are lazily loaded. |
1235 | + if base in ("core", "core16", "core18"): |
1236 | + if self.options.python_version == "python3": |
1237 | + self.build_packages.extend( |
1238 | + [ |
1239 | + "python3-dev", |
1240 | + "python3-pip", |
1241 | + "python3-pkg-resources", |
1242 | + "python3-setuptools", |
1243 | + ] |
1244 | + ) |
1245 | + elif self.options.python_version == "python2": |
1246 | + self.build_packages.extend( |
1247 | + [ |
1248 | + "python-dev", |
1249 | + "python-pip", |
1250 | + "python-pkg-resources", |
1251 | + "python-setuptools", |
1252 | + ] |
1253 | + ) |
1254 | + else: |
1255 | + raise errors.PluginBaseError(part_name=self.name, base=base) |
1256 | + |
1257 | + def pull(self): |
1258 | + super().pull() |
1259 | + |
1260 | + self._pip.setup( |
1261 | + no_index=self.options.pip_no_index, |
1262 | + find_links=self.options.pip_find_links |
1263 | + ) |
1264 | + |
1265 | + with simple_env_bzr(os.path.join(self.installdir, "bin")): |
1266 | + # Download this project, using its setup.py if present. This will |
1267 | + # also download any python-packages requested. |
1268 | + self._download_project() |
1269 | + |
1270 | + def clean_pull(self): |
1271 | + super().clean_pull() |
1272 | + self._pip.clean_packages() |
1273 | + |
1274 | + def build(self): |
1275 | + super().build() |
1276 | + |
1277 | + with simple_env_bzr(os.path.join(self.installdir, "bin")): |
1278 | + # Install the packages that have already been downloaded |
1279 | + installed_pipy_packages = self._install_project() |
1280 | + |
1281 | + requirements = self._get_list_of_packages_from_property( |
1282 | + self.options.requirements |
1283 | + ) |
1284 | + if requirements: |
1285 | + self._manifest["requirements-contents"] = requirements |
1286 | + |
1287 | + constraints = self._get_list_of_packages_from_property(self.options.constraints) |
1288 | + if constraints: |
1289 | + self._manifest["constraints-contents"] = constraints |
1290 | + |
1291 | + self._manifest["python-packages"] = [ |
1292 | + "{}={}".format(name, installed_pipy_packages[name]) |
1293 | + for name in installed_pipy_packages |
1294 | + ] |
1295 | + |
1296 | + _python.generate_sitecustomize( |
1297 | + self._python_major_version, |
1298 | + stage_dir=self.project.stage_dir, |
1299 | + install_dir=self.installdir, |
1300 | + ) |
1301 | + |
1302 | + def _find_file(self, *, filename: str) -> str: |
1303 | + # source-subdir defaults to '' |
1304 | + for basepath in [self.builddir, self.sourcedir]: |
1305 | + if basepath == self.sourcedir: |
1306 | + # This is overwritten in the base plugin |
1307 | + # TODO add consistency |
1308 | + source_subdir = self.options.source_subdir |
1309 | + else: |
1310 | + source_subdir = "" |
1311 | + filepath = os.path.join(basepath, source_subdir, filename) |
1312 | + if os.path.exists(filepath): |
1313 | + return filepath |
1314 | + |
1315 | + return None |
1316 | + |
1317 | + def _get_setup_py_dir(self): |
1318 | + setup_py_dir = None |
1319 | + setup_py_path = self._find_file(filename="setup.py") |
1320 | + if setup_py_path: |
1321 | + setup_py_dir = os.path.dirname(setup_py_path) |
1322 | + |
1323 | + return setup_py_dir |
1324 | + |
1325 | + def _get_list_of_packages_from_property(self, property_list: Set[str]) -> List[str]: |
1326 | + """Return a sorted list of all packages found in property.""" |
1327 | + package_list = list() # type: List[str] |
1328 | + for entry in property_list: |
1329 | + contents = self._get_file_contents(entry) |
1330 | + package_list.extend(contents.splitlines()) |
1331 | + return package_list |
1332 | + |
1333 | + def _get_normalized_property_set( |
1334 | + self, property_name, property_list: List[str] |
1335 | + ) -> Set[str]: |
1336 | + """Return a normalized set from a requirements or constraints list.""" |
1337 | + normalized = set() # type: Set[str] |
1338 | + for entry in property_list: |
1339 | + if isurl(entry): |
1340 | + normalized.add(entry) |
1341 | + else: |
1342 | + entry_file = self._find_file(filename=entry) |
1343 | + if not entry_file: |
1344 | + raise SnapcraftPluginPythonFileMissing( |
1345 | + plugin_property=property_name, plugin_property_value=entry |
1346 | + ) |
1347 | + normalized.add(entry_file) |
1348 | + |
1349 | + return normalized |
1350 | + |
1351 | + def _install_wheels(self, wheels): |
1352 | + installed = self._pip.list() |
1353 | + wheel_names = [os.path.basename(w).split("-")[0] for w in wheels] |
1354 | + # we want to avoid installing what is already provided in |
1355 | + # stage-packages |
1356 | + need_install = [k for k in wheel_names if k not in installed] |
1357 | + self._pip.install( |
1358 | + need_install, |
1359 | + upgrade=True, |
1360 | + install_deps=False, |
1361 | + process_dependency_links=self.options.process_dependency_links, |
1362 | + ) |
1363 | + |
1364 | + def _download_project(self): |
1365 | + constraints = self._get_normalized_property_set( |
1366 | + "constraints", self.options.constraints |
1367 | + ) |
1368 | + requirements = self._get_normalized_property_set( |
1369 | + "requirements", self.options.requirements |
1370 | + ) |
1371 | + |
1372 | + |
1373 | + self._pip.download( |
1374 | + self.options.python_packages, |
1375 | + setup_py_dir=None, |
1376 | + constraints=constraints, |
1377 | + requirements=requirements, |
1378 | + process_dependency_links=self.options.process_dependency_links, |
1379 | + no_index=self.options.pip_no_index, |
1380 | + find_links=self.options.pip_find_links, |
1381 | + ) |
1382 | + |
1383 | + def _install_project(self): |
1384 | + setup_py_dir = self._get_setup_py_dir() |
1385 | + constraints = self._get_normalized_property_set( |
1386 | + "constraints", self.options.constraints |
1387 | + ) |
1388 | + requirements = self._get_normalized_property_set( |
1389 | + "requirements", self.options.requirements |
1390 | + ) |
1391 | + |
1392 | + # setup.py is handled in a different step as some projects may |
1393 | + # need to satisfy dependencies for setup.py to be parsed. |
1394 | + wheels = self._pip.wheel( |
1395 | + self.options.python_packages, |
1396 | + setup_py_dir=None, |
1397 | + constraints=constraints, |
1398 | + requirements=requirements, |
1399 | + process_dependency_links=self.options.process_dependency_links, |
1400 | + ) |
1401 | + |
1402 | + if wheels: |
1403 | + self._install_wheels(wheels) |
1404 | + |
1405 | + if setup_py_dir is not None: |
1406 | + self._pip.download( |
1407 | + [], |
1408 | + setup_py_dir=setup_py_dir, |
1409 | + constraints=constraints, |
1410 | + requirements=set(), |
1411 | + process_dependency_links=self.options.process_dependency_links, |
1412 | + no_index=self.options.pip_no_index, |
1413 | + find_links=self.options.pip_find_links, |
1414 | + ) |
1415 | + wheels = self._pip.wheel( |
1416 | + [], |
1417 | + setup_py_dir=setup_py_dir, |
1418 | + constraints=constraints, |
1419 | + requirements=set(), |
1420 | + process_dependency_links=self.options.process_dependency_links, |
1421 | + ) |
1422 | + |
1423 | + if wheels: |
1424 | + self._install_wheels(wheels) |
1425 | + |
1426 | + setup_py_path = os.path.join(setup_py_dir, "setup.py") |
1427 | + if os.path.exists(setup_py_path): |
1428 | + # pbr and others don't work using `pip install .` |
1429 | + # LP: #1670852 |
1430 | + # There is also a chance that this setup.py is distutils based |
1431 | + # in which case we will rely on the `pip install .` ran before |
1432 | + # this. |
1433 | + with contextlib.suppress(SnapcraftPluginCommandError): |
1434 | + self._setup_tools_install(setup_py_path) |
1435 | + |
1436 | + return self._pip.list() |
1437 | + |
1438 | + def _setup_tools_install(self, setup_file): |
1439 | + command = [ |
1440 | + _python.get_python_command( |
1441 | + self._python_major_version, |
1442 | + stage_dir=self.project.stage_dir, |
1443 | + install_dir=self.installdir, |
1444 | + ), |
1445 | + os.path.basename(setup_file), |
1446 | + "--no-user-cfg", |
1447 | + "install", |
1448 | + "--single-version-externally-managed", |
1449 | + "--user", |
1450 | + "--record", |
1451 | + "install.txt", |
1452 | + ] |
1453 | + self.run(command, env=self._pip.env(), cwd=os.path.dirname(setup_file)) |
1454 | + |
1455 | + # Fix all shebangs to use the in-snap python. The stuff installed from |
1456 | + # pip has already been fixed, but anything done in this step has not. |
1457 | + mangling.rewrite_python_shebangs(self.installdir) |
1458 | + |
1459 | + def _get_file_contents(self, path): |
1460 | + if isurl(path): |
1461 | + return requests.get(path).text |
1462 | + else: |
1463 | + file_path = os.path.join(self.sourcedir, path) |
1464 | + with open(file_path) as _file: |
1465 | + return _file.read() |
1466 | + |
1467 | + def get_manifest(self): |
1468 | + return self._manifest |
1469 | + |
1470 | + def snap_fileset(self): |
1471 | + fileset = super().snap_fileset() |
1472 | + fileset.append("-bin/pip") |
1473 | + fileset.append("-bin/pip2") |
1474 | + fileset.append("-bin/pip3") |
1475 | + fileset.append("-bin/pip2.7") |
1476 | + fileset.append("-bin/pip3.*") |
1477 | + fileset.append("-bin/easy_install*") |
1478 | + fileset.append("-bin/wheel") |
1479 | + # The RECORD files include hashes useful when uninstalling packages. |
1480 | + # In the snap they will cause conflicts when more than one part uses |
1481 | + # the python plugin. |
1482 | + fileset.append("-lib/python*/site-packages/*/RECORD") |
1483 | + return fileset |
1484 | + |
1485 | + |
1486 | +@contextlib.contextmanager |
1487 | +def simple_env_bzr(bin_dir): |
1488 | + """Create an appropriate environment to run bzr. |
1489 | + |
1490 | + The python plugin sets up PYTHONUSERBASE and PYTHONHOME which |
1491 | + conflicts with bzr when using python3 as those two environment |
1492 | + variables will make bzr look for modules in the wrong location. |
1493 | + """ |
1494 | + os.makedirs(bin_dir, exist_ok=True) |
1495 | + bzr_bin = os.path.join(bin_dir, "bzr") |
1496 | + real_bzr_bin = which("bzr") |
1497 | + if real_bzr_bin: |
1498 | + exec_line = 'exec {} "$@"'.format(real_bzr_bin) |
1499 | + else: |
1500 | + exec_line = "echo bzr needs to be in PATH; exit 1" |
1501 | + with open(bzr_bin, "w") as f: |
1502 | + f.write( |
1503 | + dedent( |
1504 | + """#!/bin/sh |
1505 | + unset PYTHONUSERBASE |
1506 | + unset PYTHONHOME |
1507 | + {} |
1508 | + """.format( |
1509 | + exec_line |
1510 | + ) |
1511 | + ) |
1512 | + ) |
1513 | + os.chmod(bzr_bin, 0o777) |
1514 | + try: |
1515 | + yield |
1516 | + finally: |
1517 | + os.remove(bzr_bin) |
1518 | + if not os.listdir(bin_dir): |
1519 | + os.rmdir(bin_dir) |
1520 | diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml |
1521 | index f234ee1..613affa 100644 |
1522 | --- a/snap/snapcraft.yaml |
1523 | +++ b/snap/snapcraft.yaml |
1524 | @@ -1,10 +1,10 @@ |
1525 | + |
1526 | name: snap-store-proxy-client |
1527 | -version: 1.0 |
1528 | +version: "1.0" |
1529 | summary: Canonical snap store proxy administration client. |
1530 | description: | |
1531 | The Canonical snapstore client is used to manage a snap store proxy. |
1532 | - |
1533 | -# Defaults, but snapcraft prints ugly yellow messages without them. |
1534 | +base: core |
1535 | confinement: strict |
1536 | grade: stable |
1537 | |
1538 | @@ -16,17 +16,24 @@ apps: |
1539 | |
1540 | parts: |
1541 | deps: |
1542 | - plugin: python |
1543 | - requirements: ./requirements.txt |
1544 | + plugin: python-ols |
1545 | + source: . |
1546 | + pip-find-links: |
1547 | + - $SNAPCRAFT_PROJECT_DIR/dependencies |
1548 | + pip-no-index: true |
1549 | + requirements: |
1550 | + - ./requirements.txt |
1551 | stage-packages: |
1552 | - libsodium18 |
1553 | source: |
1554 | plugin: dump |
1555 | source: . |
1556 | + override-build: | |
1557 | + #TODO: Would be a lot simpler to make snapstore_client normal |
1558 | + # Python package |
1559 | + cp -a $SNAPCRAFT_PROJECT_DIR/snapstore $SNAPCRAFT_PART_INSTALL |
1560 | + cp -a $SNAPCRAFT_PROJECT_DIR/snapstore_client $SNAPCRAFT_PART_INSTALL |
1561 | stage: |
1562 | - ./snapstore |
1563 | - ./snapstore_client/ |
1564 | - override-prime: | |
1565 | - snapcraftctl prime |
1566 | - /snap/core/current/usr/bin/python3 -m compileall -q -j0 snapstore_client |
1567 | |
1568 | diff --git a/snapstore b/snapstore |
1569 | index cd67b71..4550e58 100755 |
1570 | --- a/snapstore |
1571 | +++ b/snapstore |
1572 | @@ -1,132 +1,7 @@ |
1573 | #!/usr/bin/env python3 |
1574 | -# -*- coding: utf-8 -*- |
1575 | -# Copyright 2017 Canonical Ltd. This software is licensed under the |
1576 | -# GNU General Public License version 3 (see the file LICENSE). |
1577 | - |
1578 | -"""CLI utility to manage a store-in-a-box installation.""" |
1579 | - |
1580 | -import argparse |
1581 | -import logging |
1582 | -import os |
1583 | import sys |
1584 | |
1585 | -from snapstore_client.cli import configure_logging |
1586 | -from snapstore_client.logic.login import login |
1587 | -from snapstore_client.logic.overrides import ( |
1588 | - delete_override, |
1589 | - list_overrides, |
1590 | - override, |
1591 | -) |
1592 | -from snapstore_client.logic.push import push_snap |
1593 | - |
1594 | - |
1595 | -DEFAULT_SSO_URL = 'https://login.ubuntu.com/' |
1596 | - |
1597 | - |
1598 | -def main(): |
1599 | - try: |
1600 | - configure_logging() |
1601 | - args = parse_args() |
1602 | - return args.func(args) or 0 |
1603 | - except KeyboardInterrupt: |
1604 | - # use default logger as we can't guarantee that configuration |
1605 | - # has completed. |
1606 | - logging.error("Operation cancelled") |
1607 | - return 1 |
1608 | - |
1609 | - |
1610 | -def parse_args(): |
1611 | - parser = argparse.ArgumentParser() |
1612 | - subparsers = parser.add_subparsers(help='sub-command help') |
1613 | - |
1614 | - login_parser = subparsers.add_parser( |
1615 | - 'login', help='Sign into a store.', |
1616 | - formatter_class=argparse.ArgumentDefaultsHelpFormatter) |
1617 | - login_parser.add_argument('store_url', help='Store URL') |
1618 | - login_parser.add_argument('email', help='Ubuntu One SSO email', nargs='?') |
1619 | - login_parser.add_argument('--sso-url', help='Ubuntu One SSO URL', |
1620 | - default=DEFAULT_SSO_URL) |
1621 | - login_parser.add_argument('--offline', help="Use offline mode interaction", |
1622 | - action='store_true') |
1623 | - login_parser.set_defaults(func=login) |
1624 | - |
1625 | - list_overrides_parser = subparsers.add_parser( |
1626 | - 'list-overrides', help='List channel map overrides.', |
1627 | - ) |
1628 | - list_overrides_parser.add_argument( |
1629 | - '--series', default='16', |
1630 | - help='The series within which to list overrides.') |
1631 | - list_overrides_parser.add_argument( |
1632 | - 'snap_name', |
1633 | - help='The name of the snap whose channel map should be listed.') |
1634 | - list_overrides_parser.add_argument( |
1635 | - '--password', |
1636 | - help='Password for interacting with an offline proxy', |
1637 | - default=os.environ.get('SNAP_PROXY_PASSWORD') |
1638 | - ) |
1639 | - list_overrides_parser.set_defaults(func=list_overrides) |
1640 | - |
1641 | - override_parser = subparsers.add_parser( |
1642 | - 'override', help='Set channel map overrides.', |
1643 | - ) |
1644 | - override_parser.add_argument( |
1645 | - '--series', default='16', |
1646 | - help='The series within which to set overrides.') |
1647 | - override_parser.add_argument( |
1648 | - 'snap_name', |
1649 | - help='The name of the snap whose channel map should be modified.') |
1650 | - override_parser.add_argument( |
1651 | - 'channel_map_entries', nargs='+', metavar='channel_map_entry', |
1652 | - help='A channel map override, in the form <channel>=<revision>.') |
1653 | - override_parser.add_argument( |
1654 | - '--password', |
1655 | - help='Password for interacting with an offline proxy', |
1656 | - default=os.environ.get('SNAP_PROXY_PASSWORD') |
1657 | - ) |
1658 | - override_parser.set_defaults(func=override) |
1659 | - |
1660 | - delete_override_parser = subparsers.add_parser( |
1661 | - 'delete-override', help='Delete channel map overrides.', |
1662 | - ) |
1663 | - delete_override_parser.add_argument( |
1664 | - '--series', default='16', |
1665 | - help='The series within which to delete overrides.') |
1666 | - delete_override_parser.add_argument( |
1667 | - 'snap_name', |
1668 | - help='The name of the snap whose channel map should be modified.') |
1669 | - delete_override_parser.add_argument( |
1670 | - 'channels', nargs='+', metavar='channel', |
1671 | - help='A channel whose overrides should be deleted.') |
1672 | - delete_override_parser.add_argument( |
1673 | - '--password', |
1674 | - help='Password for interacting with an offline proxy', |
1675 | - default=os.environ.get('SNAP_PROXY_PASSWORD') |
1676 | - ) |
1677 | - delete_override_parser.set_defaults(func=delete_override) |
1678 | - |
1679 | - push_snap_parser = subparsers.add_parser( |
1680 | - 'push-snap', help='push a snap to an offline proxy') |
1681 | - push_snap_parser.add_argument( |
1682 | - 'snap_tar_file', |
1683 | - help='The .tar.gz file of a bundled downloaded snap') |
1684 | - push_snap_parser.add_argument( |
1685 | - '--push-channel-map', |
1686 | - action='store_true', |
1687 | - help="Force push of the channel map," |
1688 | - " removing any existing overrides") |
1689 | - push_snap_parser.add_argument( |
1690 | - '--password', |
1691 | - help='Password for interacting with an offline proxy', |
1692 | - default=os.environ.get('SNAP_PROXY_PASSWORD') |
1693 | - ) |
1694 | - push_snap_parser.set_defaults(func=push_snap) |
1695 | - |
1696 | - if len(sys.argv) == 1: |
1697 | - # Display help if no arguments are provided. |
1698 | - parser.print_help() |
1699 | - sys.exit(1) |
1700 | - |
1701 | - return parser.parse_args() |
1702 | +from snapstore_client.__main__ import main |
1703 | |
1704 | |
1705 | if __name__ == '__main__': |
1706 | diff --git a/snapstore_client/__main__.py b/snapstore_client/__main__.py |
1707 | new file mode 100644 |
1708 | index 0000000..be6029a |
1709 | --- /dev/null |
1710 | +++ b/snapstore_client/__main__.py |
1711 | @@ -0,0 +1,136 @@ |
1712 | +# Copyright 2017 Canonical Ltd. This software is licensed under the |
1713 | +# GNU General Public License version 3 (see the file LICENSE). |
1714 | + |
1715 | +"""CLI utility to manage a store-in-a-box installation.""" |
1716 | + |
1717 | +import argparse |
1718 | +import logging |
1719 | +import os |
1720 | +import sys |
1721 | + |
1722 | +from snapstore_client.cli import configure_logging |
1723 | +from snapstore_client.logic.login import login |
1724 | +from snapstore_client.logic.overrides import ( |
1725 | + delete_override, |
1726 | + list_overrides, |
1727 | + override, |
1728 | +) |
1729 | +from snapstore_client.logic.push import push_snap |
1730 | + |
1731 | + |
1732 | +logger = logging.getLogger(__name__) |
1733 | + |
1734 | + |
1735 | +DEFAULT_SSO_URL = 'https://login.ubuntu.com/' |
1736 | + |
1737 | + |
1738 | +def main(): |
1739 | + try: |
1740 | + configure_logging() |
1741 | + args = parse_args() |
1742 | + return args.func(args) or 0 |
1743 | + except Exception as e: |
1744 | + logging.error("%s: %s", type(e).__name__, e) |
1745 | + except KeyboardInterrupt: |
1746 | + # use default logger as we can't guarantee that configuration |
1747 | + # has completed. |
1748 | + logging.error("Operation cancelled") |
1749 | + return 1 |
1750 | + |
1751 | + |
1752 | +def parse_args(): |
1753 | + parser = argparse.ArgumentParser() |
1754 | + subparsers = parser.add_subparsers(help='sub-command help') |
1755 | + |
1756 | + login_parser = subparsers.add_parser( |
1757 | + 'login', help='Sign into a store.', |
1758 | + formatter_class=argparse.ArgumentDefaultsHelpFormatter) |
1759 | + login_parser.add_argument('store_url', help='Store URL') |
1760 | + login_parser.add_argument('email', help='Ubuntu One SSO email', nargs='?') |
1761 | + login_parser.add_argument('--sso-url', help='Ubuntu One SSO URL', |
1762 | + default=DEFAULT_SSO_URL) |
1763 | + login_parser.add_argument('--offline', help="Use offline mode interaction", |
1764 | + action='store_true') |
1765 | + login_parser.set_defaults(func=login) |
1766 | + |
1767 | + list_overrides_parser = subparsers.add_parser( |
1768 | + 'list-overrides', help='List channel map overrides.', |
1769 | + ) |
1770 | + list_overrides_parser.add_argument( |
1771 | + '--series', default='16', |
1772 | + help='The series within which to list overrides.') |
1773 | + list_overrides_parser.add_argument( |
1774 | + 'snap_name', |
1775 | + help='The name of the snap whose channel map should be listed.') |
1776 | + list_overrides_parser.add_argument( |
1777 | + '--password', |
1778 | + help='Password for interacting with an offline proxy', |
1779 | + default=os.environ.get('SNAP_PROXY_PASSWORD') |
1780 | + ) |
1781 | + list_overrides_parser.set_defaults(func=list_overrides) |
1782 | + |
1783 | + override_parser = subparsers.add_parser( |
1784 | + 'override', help='Set channel map overrides.', |
1785 | + ) |
1786 | + override_parser.add_argument( |
1787 | + '--series', default='16', |
1788 | + help='The series within which to set overrides.') |
1789 | + override_parser.add_argument( |
1790 | + 'snap_name', |
1791 | + help='The name of the snap whose channel map should be modified.') |
1792 | + override_parser.add_argument( |
1793 | + 'channel_map_entries', nargs='+', metavar='channel_map_entry', |
1794 | + help='A channel map override, in the form <channel>=<revision>.') |
1795 | + override_parser.add_argument( |
1796 | + '--password', |
1797 | + help='Password for interacting with an offline proxy', |
1798 | + default=os.environ.get('SNAP_PROXY_PASSWORD') |
1799 | + ) |
1800 | + override_parser.set_defaults(func=override) |
1801 | + |
1802 | + delete_override_parser = subparsers.add_parser( |
1803 | + 'delete-override', help='Delete channel map overrides.', |
1804 | + ) |
1805 | + delete_override_parser.add_argument( |
1806 | + '--series', default='16', |
1807 | + help='The series within which to delete overrides.') |
1808 | + delete_override_parser.add_argument( |
1809 | + 'snap_name', |
1810 | + help='The name of the snap whose channel map should be modified.') |
1811 | + delete_override_parser.add_argument( |
1812 | + 'channels', nargs='+', metavar='channel', |
1813 | + help='A channel whose overrides should be deleted.') |
1814 | + delete_override_parser.add_argument( |
1815 | + '--password', |
1816 | + help='Password for interacting with an offline proxy', |
1817 | + default=os.environ.get('SNAP_PROXY_PASSWORD') |
1818 | + ) |
1819 | + delete_override_parser.set_defaults(func=delete_override) |
1820 | + |
1821 | + push_snap_parser = subparsers.add_parser( |
1822 | + 'push-snap', help='push a snap to an offline proxy') |
1823 | + push_snap_parser.add_argument( |
1824 | + 'snap_tar_file', |
1825 | + help='The .tar.gz file of a bundled downloaded snap') |
1826 | + push_snap_parser.add_argument( |
1827 | + '--push-channel-map', |
1828 | + action='store_true', |
1829 | + help="Force push of the channel map," |
1830 | + " removing any existing overrides") |
1831 | + push_snap_parser.add_argument( |
1832 | + '--password', |
1833 | + help='Password for interacting with an offline proxy', |
1834 | + default=os.environ.get('SNAP_PROXY_PASSWORD') |
1835 | + ) |
1836 | + push_snap_parser.set_defaults(func=push_snap) |
1837 | + |
1838 | + if len(sys.argv) == 1: |
1839 | + # Display help if no arguments are provided. |
1840 | + parser.print_help() |
1841 | + sys.exit(1) |
1842 | + |
1843 | + return parser.parse_args() |
1844 | + |
1845 | + |
1846 | +if __name__ == '__main__': |
1847 | + sys.exit(main()) |
1848 | diff --git a/snapstore_client/exceptions.py b/snapstore_client/exceptions.py |
1849 | deleted file mode 100644 |
1850 | index 061b5fe..0000000 |
1851 | --- a/snapstore_client/exceptions.py |
1852 | +++ /dev/null |
1853 | @@ -1,72 +0,0 @@ |
1854 | -# Copyright 2017 Canonical Ltd. This software is licensed under the |
1855 | -# GNU General Public License version 3 (see the file LICENSE). |
1856 | - |
1857 | -"""Client exceptions.""" |
1858 | - |
1859 | - |
1860 | -class ClientError(Exception): |
1861 | - """The base class for all user-visible exceptions.""" |
1862 | - |
1863 | - @property |
1864 | - def fmt(self): |
1865 | - raise NotImplementedError |
1866 | - |
1867 | - def __init__(self, **kwargs): |
1868 | - for key, value in kwargs.items(): |
1869 | - setattr(self, key, value) |
1870 | - |
1871 | - def __str__(self): |
1872 | - return self.fmt.format(**self.__dict__) |
1873 | - |
1874 | - |
1875 | -class InvalidCredentials(ClientError): |
1876 | - |
1877 | - fmt = 'Invalid credentials: {message}.' |
1878 | - |
1879 | - def __init__(self, message): |
1880 | - super().__init__(message=message) |
1881 | - |
1882 | - |
1883 | -class InvalidStoreURL(ClientError): |
1884 | - |
1885 | - fmt = 'Invalid store url: {message}.' |
1886 | - |
1887 | - def __init__(self, message): |
1888 | - super().__init__(message=message) |
1889 | - |
1890 | - |
1891 | -class StoreCommunicationError(ClientError): |
1892 | - |
1893 | - fmt = 'Connection error with the store using: {message}.' |
1894 | - |
1895 | - def __init__(self, message): |
1896 | - super().__init__(message=message) |
1897 | - |
1898 | - |
1899 | -class StoreMacaroonSSOMismatch(ClientError): |
1900 | - |
1901 | - fmt = 'Root macaroon does not refer to expected SSO host: {sso_host}.' |
1902 | - |
1903 | - def __init__(self, sso_host): |
1904 | - super().__init__(sso_host=sso_host) |
1905 | - |
1906 | - |
1907 | -class StoreAuthenticationError(ClientError): |
1908 | - |
1909 | - # No terminating full stop because the message from SSO sometimes |
1910 | - # (though not always!) includes one. |
1911 | - fmt = 'Authentication error: {message}' |
1912 | - |
1913 | - def __init__(self, message, extra=None): |
1914 | - super().__init__(message=message, extra=extra) |
1915 | - |
1916 | - |
1917 | -class StoreTwoFactorAuthenticationRequired(StoreAuthenticationError): |
1918 | - |
1919 | - def __init__(self): |
1920 | - super().__init__('Two-factor authentication required.') |
1921 | - |
1922 | - |
1923 | -class StoreMacaroonNeedsRefresh(ClientError): |
1924 | - |
1925 | - fmt = 'Authentication macaroon needs to be refreshed.' |
1926 | diff --git a/snapstore_client/logic/login.py b/snapstore_client/logic/login.py |
1927 | index 9550988..1eb92ef 100644 |
1928 | --- a/snapstore_client/logic/login.py |
1929 | +++ b/snapstore_client/logic/login.py |
1930 | @@ -2,86 +2,63 @@ |
1931 | |
1932 | import getpass |
1933 | import logging |
1934 | -from urllib.parse import urlparse |
1935 | +from urllib.parse import urljoin |
1936 | |
1937 | -from pymacaroons import Macaroon |
1938 | +import requests |
1939 | |
1940 | -from snapstore_client import ( |
1941 | - config, |
1942 | - exceptions, |
1943 | - webservices as ws, |
1944 | -) |
1945 | +from snapstore_client import config |
1946 | +from snapstore_library import exceptions, authentication |
1947 | |
1948 | |
1949 | logger = logging.getLogger(__name__) |
1950 | |
1951 | |
1952 | -def _extract_caveat_id(sso_url, root_macaroon): |
1953 | - macaroon = Macaroon.deserialize(root_macaroon) |
1954 | - sso_host = urlparse(sso_url).netloc |
1955 | - for caveat in macaroon.caveats: |
1956 | - if caveat.location == sso_host: |
1957 | - return caveat.caveat_id |
1958 | - else: |
1959 | - raise exceptions.StoreMacaroonSSOMismatch(sso_host) |
1960 | +def set_offline(args): |
1961 | + store_name = "default" |
1962 | + cfg = config.Config() |
1963 | + store = cfg.store_section(store_name) |
1964 | + store.set("gw_url", args.store_url) |
1965 | + cfg.save() |
1966 | + logger.info("Configured %s as the %s store", args.store_url, store_name) |
1967 | + return 0 |
1968 | |
1969 | |
1970 | def login(args): |
1971 | - # TODO: validate these before using to avoid ugly errors. |
1972 | - gw_url = args.store_url |
1973 | - sso_url = args.sso_url |
1974 | - |
1975 | if args.offline: |
1976 | - cfg = config.Config() |
1977 | - store = cfg.store_section('default') |
1978 | - store.set('gw_url', gw_url) |
1979 | - cfg.save() |
1980 | - return |
1981 | + return set_offline(args) |
1982 | |
1983 | - logger.info('Enter your Ubuntu One SSO credentials.') |
1984 | + logger.info("Enter your Ubuntu One SSO credentials.") |
1985 | email = args.email |
1986 | if not email: |
1987 | - email = input('Email: ') |
1988 | - password = getpass.getpass('Password: ') |
1989 | + email = input("Email: ") |
1990 | + password = getpass.getpass("Password: ") |
1991 | |
1992 | + session = requests.Session() |
1993 | + auth_url = urljoin(args.store_url, "/v2/auth/issue-store-admin") |
1994 | + sso_url = urljoin(args.sso_url, "/api/v2/tokens/discharge") |
1995 | try: |
1996 | - root = ws.issue_store_admin(gw_url) |
1997 | - except exceptions.ClientError as e: |
1998 | - logger.error(str(e)) |
1999 | - return |
2000 | - caveat_id = _extract_caveat_id(sso_url, root) |
2001 | - try: |
2002 | - try: |
2003 | - unbound_discharge = ws.get_sso_discharge( |
2004 | - sso_url, email, password, caveat_id) |
2005 | - logger.info('Login successful') |
2006 | - except exceptions.StoreTwoFactorAuthenticationRequired: |
2007 | - one_time_password = input('Second-factor auth: ') |
2008 | - unbound_discharge = ws.get_sso_discharge( |
2009 | - sso_url, email, password, caveat_id, |
2010 | - one_time_password=one_time_password) |
2011 | - logger.info('Login successful') |
2012 | - except exceptions.StoreAuthenticationError as e: |
2013 | - logger.error('Login failed.') |
2014 | - logger.error('%s', e) |
2015 | - if e.extra: |
2016 | - for key, value in e.extra.items(): |
2017 | - if isinstance(value, list): |
2018 | - value = ' '.join(value) |
2019 | - logger.error('%s: %s', key, value) |
2020 | - return 1 |
2021 | + root_macaroon, discharge_macaroon = authentication.login( |
2022 | + session, auth_url, sso_url, email, password |
2023 | + ) |
2024 | + except exceptions.TwoFactorAuthenticationRequired: |
2025 | + one_time_password = input("Second-factor auth: ") |
2026 | + root_macaroon, discharge_macaroon = authentication.login( |
2027 | + session, auth_url, sso_url, email, password, otp=one_time_password |
2028 | + ) |
2029 | + logger.info("Login successful") |
2030 | |
2031 | cfg = config.Config() |
2032 | # For now, the store is always called "default". In the future we may want |
2033 | # to support multiple stores by allowing the user to provide a nice name |
2034 | # for a store at login that can be used to select the store for later |
2035 | # operations. |
2036 | - store = cfg.store_section('default') |
2037 | - store.set('gw_url', gw_url) |
2038 | - store.set('sso_url', sso_url) |
2039 | - store.set('root', root) |
2040 | - store.set('unbound_discharge', unbound_discharge) |
2041 | - store.set('email', email) |
2042 | + store_name = "default" |
2043 | + store = cfg.store_section(store_name) |
2044 | + store.set("gw_url", args.store_url) |
2045 | + store.set("sso_url", args.sso_url) |
2046 | + store.set("root", root_macaroon.serialize()) |
2047 | + store.set("unbound_discharge", discharge_macaroon.serialize()) |
2048 | + store.set("email", email) |
2049 | cfg.save() |
2050 | - |
2051 | + logger.info("Configured %s as the %s store", args.store_url, store_name) |
2052 | return 0 |
2053 | diff --git a/snapstore_client/logic/overrides.py b/snapstore_client/logic/overrides.py |
2054 | index d3c8356..aa6d490 100644 |
2055 | --- a/snapstore_client/logic/overrides.py |
2056 | +++ b/snapstore_client/logic/overrides.py |
2057 | @@ -1,51 +1,73 @@ |
2058 | # Copyright 2017 Canonical Ltd. |
2059 | |
2060 | import logging |
2061 | +from urllib.parse import urljoin |
2062 | |
2063 | -from requests.exceptions import HTTPError |
2064 | +import requests |
2065 | |
2066 | -from snapstore_client import ( |
2067 | - config, |
2068 | - exceptions, |
2069 | - webservices as ws, |
2070 | -) |
2071 | +from snapstore_client import config |
2072 | from snapstore_client.presentation_helpers import ( |
2073 | channel_map_string_to_tuple, |
2074 | override_to_string, |
2075 | ) |
2076 | from snapstore_client.utils import ( |
2077 | _check_default_store, |
2078 | - _log_authorized_error, |
2079 | - _log_credentials_error, |
2080 | ) |
2081 | +from snapstore_library import authentication, overrides |
2082 | |
2083 | |
2084 | logger = logging.getLogger(__name__) |
2085 | |
2086 | |
2087 | +# Used by the offline store |
2088 | +HARDCODED_ADMIN = "admin" |
2089 | + |
2090 | + |
2091 | +def _get_macaroon_auth(session, store): |
2092 | + refresh_url = urljoin(store.get("sso_url"), "/api/v2/tokens/refresh") |
2093 | + refresh_handler = authentication.MacaroonRefreshHandler( |
2094 | + session, refresh_url |
2095 | + ) |
2096 | + |
2097 | + def discharge_updater(discharge): |
2098 | + store.set("unbound_discharge", discharge) |
2099 | + |
2100 | + return authentication.HTTPMacaroonAuth( |
2101 | + authentication.deserialize(store.get("root")), |
2102 | + authentication.deserialize(store.get("unbound_discharge")), |
2103 | + refresh_handler=refresh_handler, |
2104 | + dischage_update_handler=discharge_updater, |
2105 | + ) |
2106 | + |
2107 | + |
2108 | +def _get_auth(session, args, store): |
2109 | + if args.password: |
2110 | + auth = requests.auth.HTTPBasicAuth(HARDCODED_ADMIN, args.password) |
2111 | + else: |
2112 | + auth = _get_macaroon_auth(session, store) |
2113 | + return auth |
2114 | + |
2115 | + |
2116 | +def _print_overrides(overrides_response): |
2117 | + for override in overrides_response["overrides"]: |
2118 | + logger.info(override_to_string(override)) |
2119 | + |
2120 | + |
2121 | def list_overrides(args): |
2122 | cfg = config.Config() |
2123 | store = _check_default_store(cfg) |
2124 | if not store: |
2125 | return 1 |
2126 | |
2127 | - try: |
2128 | - if args.password: |
2129 | - response = ws.get_overrides( |
2130 | - store, args.snap_name, |
2131 | - series=args.series, password=args.password) |
2132 | - else: |
2133 | - response = ws.refresh_if_necessary( |
2134 | - store, ws.get_overrides, |
2135 | - store, args.snap_name, series=args.series) |
2136 | - except exceptions.InvalidCredentials as e: |
2137 | - _log_credentials_error(e) |
2138 | - return 1 |
2139 | - except HTTPError: |
2140 | - _log_authorized_error() |
2141 | - return 1 |
2142 | - for override in response['overrides']: |
2143 | - logger.info(override_to_string(override)) |
2144 | + session = requests.Session() |
2145 | + auth = _get_auth(session, args, store) |
2146 | + url = urljoin( |
2147 | + store.get("gw_url"), "/v2/metadata/overrides/{}".format(args.snap_name) |
2148 | + ) |
2149 | + overrides_response = overrides.get_overrides( |
2150 | + session, url, args.snap_name, auth=auth |
2151 | + ) |
2152 | + _print_overrides(overrides_response) |
2153 | |
2154 | |
2155 | def override(args): |
2156 | @@ -54,31 +76,24 @@ def override(args): |
2157 | if not store: |
2158 | return 1 |
2159 | |
2160 | - overrides = [] |
2161 | + overrides_data = [] |
2162 | for channel_map_entry in args.channel_map_entries: |
2163 | channel, revision = channel_map_string_to_tuple(channel_map_entry) |
2164 | - overrides.append({ |
2165 | - 'snap_name': args.snap_name, |
2166 | - 'revision': revision, |
2167 | - 'channel': channel, |
2168 | - 'series': args.series, |
2169 | - }) |
2170 | - try: |
2171 | - if args.password: |
2172 | - response = ws.set_overrides( |
2173 | - store, overrides, password=args.password) |
2174 | - else: |
2175 | - response = ws.refresh_if_necessary( |
2176 | - store, ws.set_overrides, |
2177 | - store, overrides) |
2178 | - except exceptions.InvalidCredentials as e: |
2179 | - _log_credentials_error(e) |
2180 | - return 1 |
2181 | - except HTTPError: |
2182 | - _log_authorized_error() |
2183 | - return 1 |
2184 | - for override in response['overrides']: |
2185 | - logger.info(override_to_string(override)) |
2186 | + overrides_data.append( |
2187 | + { |
2188 | + "snap_name": args.snap_name, |
2189 | + "revision": revision, |
2190 | + "channel": channel, |
2191 | + "series": args.series, |
2192 | + } |
2193 | + ) |
2194 | + session = requests.Session() |
2195 | + auth = _get_auth(session, args, store) |
2196 | + url = urljoin(store.get("gw_url"), "/v2/metadata/overrides") |
2197 | + overrides_response = overrides.set_overrides( |
2198 | + session, url, overrides_data, auth=auth |
2199 | + ) |
2200 | + _print_overrides(overrides_response) |
2201 | |
2202 | |
2203 | def delete_override(args): |
2204 | @@ -87,27 +102,20 @@ def delete_override(args): |
2205 | if not store: |
2206 | return 1 |
2207 | |
2208 | - overrides = [] |
2209 | + overrides_data = [] |
2210 | for channel in args.channels: |
2211 | - overrides.append({ |
2212 | - 'snap_name': args.snap_name, |
2213 | - 'revision': None, |
2214 | - 'channel': channel, |
2215 | - 'series': args.series, |
2216 | - }) |
2217 | - try: |
2218 | - if args.password: |
2219 | - response = ws.set_overrides( |
2220 | - store, overrides, password=args.password) |
2221 | - else: |
2222 | - response = ws.refresh_if_necessary( |
2223 | - store, ws.set_overrides, |
2224 | - store, overrides) |
2225 | - except exceptions.InvalidCredentials as e: |
2226 | - _log_credentials_error(e) |
2227 | - return 1 |
2228 | - except HTTPError: |
2229 | - _log_authorized_error() |
2230 | - return 1 |
2231 | - for override in response['overrides']: |
2232 | - logger.info(override_to_string(override)) |
2233 | + overrides_data.append( |
2234 | + { |
2235 | + "snap_name": args.snap_name, |
2236 | + "revision": None, |
2237 | + "channel": channel, |
2238 | + "series": args.series, |
2239 | + } |
2240 | + ) |
2241 | + session = requests.Session() |
2242 | + auth = _get_auth(session, args, store) |
2243 | + url = urljoin(store.get("gw_url"), "/v2/metadata/overrides") |
2244 | + overrides_response = overrides.set_overrides( |
2245 | + session, url, overrides_data, auth=auth |
2246 | + ) |
2247 | + _print_overrides(overrides_response) |
2248 | diff --git a/snapstore_client/logic/push.py b/snapstore_client/logic/push.py |
2249 | index abd06e8..be5fcd4 100644 |
2250 | --- a/snapstore_client/logic/push.py |
2251 | +++ b/snapstore_client/logic/push.py |
2252 | @@ -10,12 +10,10 @@ from requests.exceptions import HTTPError |
2253 | |
2254 | from snapstore_client import ( |
2255 | config, |
2256 | - exceptions, |
2257 | ) |
2258 | from snapstore_client.utils import ( |
2259 | _check_default_store, |
2260 | _log_authorized_error, |
2261 | - _log_credentials_error, |
2262 | ) |
2263 | |
2264 | logger = logging.getLogger(__name__) |
2265 | @@ -298,9 +296,6 @@ def push_snap(args): |
2266 | logger.error( |
2267 | "This command only works with a supplied password" |
2268 | " and a proxy in offline mode") |
2269 | - except exceptions.InvalidCredentials as e: |
2270 | - _log_credentials_error(e) |
2271 | - return 1 |
2272 | except HTTPError: |
2273 | _log_authorized_error() |
2274 | return 1 |
2275 | diff --git a/snapstore_client/logic/tests/test_login.py b/snapstore_client/logic/tests/test_login.py |
2276 | index d7854cc..f2fce6e 100644 |
2277 | --- a/snapstore_client/logic/tests/test_login.py |
2278 | +++ b/snapstore_client/logic/tests/test_login.py |
2279 | @@ -1,286 +1,124 @@ |
2280 | # Copyright 2017 Canonical Ltd. |
2281 | +from testtools.assertions import assert_that |
2282 | +from testtools.matchers import Equals |
2283 | +import pymacaroons |
2284 | |
2285 | -import json |
2286 | -from unittest import mock |
2287 | -from urllib.parse import urljoin, urlparse |
2288 | +from snapstore_client.logic.tests.utils import RunMainTestCase |
2289 | +from snapstore_library import exceptions |
2290 | |
2291 | -import fixtures |
2292 | -from pymacaroons import Macaroon |
2293 | -import responses |
2294 | -from testtools import TestCase |
2295 | -from testtools.matchers import ( |
2296 | - ContainsDict, |
2297 | - Equals, |
2298 | - MatchesDict, |
2299 | -) |
2300 | |
2301 | -from snapstore_client import ( |
2302 | - config, |
2303 | - exceptions, |
2304 | -) |
2305 | -from snapstore_client.logic.login import login |
2306 | -from snapstore_client.tests import factory |
2307 | - |
2308 | - |
2309 | -class LoginTests(TestCase): |
2310 | - |
2311 | - def setUp(self): |
2312 | - super().setUp() |
2313 | - self.default_gw_url = 'http://store.local/' |
2314 | - self.default_sso_url = 'https://login.staging.ubuntu.com/' |
2315 | - self.logger = self.useFixture(fixtures.FakeLogger()) |
2316 | - self.config_path = self.useFixture(fixtures.TempDir()).path |
2317 | - self.useFixture(fixtures.MonkeyPatch( |
2318 | - 'xdg.BaseDirectory.xdg_config_home', self.config_path)) |
2319 | - self.useFixture(fixtures.MonkeyPatch( |
2320 | - 'xdg.BaseDirectory.xdg_config_dirs', [self.config_path])) |
2321 | - self.mock_input = self.useFixture(fixtures.MockPatch( |
2322 | - 'builtins.input')).mock |
2323 | - self.mock_getpass = self.useFixture(fixtures.MockPatch( |
2324 | - 'getpass.getpass')).mock |
2325 | - |
2326 | - def make_responses_callback(self, response_templates): |
2327 | - full_responses = [] |
2328 | - for response in response_templates: |
2329 | - status = response.get('status', 200) |
2330 | - content_type = 'text/plain' |
2331 | - if 'json' in response: |
2332 | - content_type = 'application/json' |
2333 | - body = json.dumps(response['json']) |
2334 | - else: |
2335 | - body = response.get('body') |
2336 | - full_responses.append( |
2337 | - (status, {'Content-Type': content_type}, body)) |
2338 | - iter_responses = iter(full_responses) |
2339 | - return lambda request: next(iter_responses) |
2340 | - |
2341 | - def make_args(self, store_url=None, |
2342 | - sso_url=None, email=None, offline=False): |
2343 | - return factory.Args( |
2344 | - store_url=store_url or self.default_gw_url, |
2345 | - sso_url=sso_url or self.default_sso_url, |
2346 | - email=email, |
2347 | - offline=offline, |
2348 | +class LoginTests(RunMainTestCase): |
2349 | + def test_offline_login(self): |
2350 | + self.mock_with_side_effect( |
2351 | + "snapstore_library.authentication.login", |
2352 | + AssertionError("should not be called"), |
2353 | ) |
2354 | |
2355 | - def add_issue_store_admin_response(self, *response_templates, gw_url=None): |
2356 | - gw_url = gw_url or self.default_gw_url |
2357 | - issue_store_admin_url = urljoin(gw_url, '/v2/auth/issue-store-admin') |
2358 | - responses.add_callback( |
2359 | - 'POST', issue_store_admin_url, |
2360 | - self.make_responses_callback(response_templates)) |
2361 | - |
2362 | - def add_get_sso_discharge_response(self, *response_templates, |
2363 | - sso_url=None): |
2364 | - sso_url = sso_url or self.default_sso_url |
2365 | - discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge') |
2366 | - responses.add_callback( |
2367 | - 'POST', discharge_url, |
2368 | - self.make_responses_callback(response_templates)) |
2369 | - |
2370 | - def make_root_macaroon(self, sso_url=None): |
2371 | - macaroon = Macaroon() |
2372 | - sso_url = sso_url or self.default_sso_url |
2373 | - sso_host = urlparse(sso_url).netloc |
2374 | - macaroon.add_third_party_caveat(sso_host, 'key', 'payload') |
2375 | - return macaroon.serialize() |
2376 | + exit_status = self.run_main( |
2377 | + "login", "http://store.local/", "--offline" |
2378 | + ) |
2379 | |
2380 | - @responses.activate |
2381 | - def test_login_sso_mismatch(self): |
2382 | - self.mock_input.return_value = 'user@example.org' |
2383 | - self.mock_getpass.return_value = 'secret' |
2384 | - macaroon = Macaroon() |
2385 | - macaroon.add_third_party_caveat('another.example.com', '', '') |
2386 | - self.add_issue_store_admin_response( |
2387 | - {'status': 200, 'json': {'macaroon': macaroon.serialize()}}) |
2388 | - self.add_get_sso_discharge_response({'status': 401}) |
2389 | - self.assertRaises( |
2390 | - exceptions.StoreMacaroonSSOMismatch, login, self.make_args()) |
2391 | + assert_that(exit_status, Equals(0)) |
2392 | + assert_that( |
2393 | + self.logger.output, |
2394 | + Equals("Configured http://store.local/ as the default store\n"), |
2395 | + ) |
2396 | |
2397 | - @responses.activate |
2398 | - def test_login_sso_bad_email(self): |
2399 | - self.mock_input.return_value = '' |
2400 | - self.mock_getpass.return_value = '' |
2401 | - self.add_issue_store_admin_response( |
2402 | - {'status': 200, 'json': {'macaroon': self.make_root_macaroon()}}) |
2403 | - auth_error = { |
2404 | - 'message': 'Invalid request data', |
2405 | - 'extra': {'email': ['Enter a valid email address.']}, |
2406 | - } |
2407 | - self.add_get_sso_discharge_response( |
2408 | - {'status': 401, 'json': {'error_list': [auth_error]}}) |
2409 | - self.assertEqual(1, login(self.make_args())) |
2410 | - self.assertEqual( |
2411 | - 'Enter your Ubuntu One SSO credentials.\n' |
2412 | - 'Login failed.\n' |
2413 | - 'Authentication error: Invalid request data\n' |
2414 | - 'email: Enter a valid email address.\n', |
2415 | - self.logger.output) |
2416 | + def test_login(self): |
2417 | + root_macaroon = pymacaroons.Macaroon(location="test1") |
2418 | + discharge_macaroon = pymacaroons.Macaroon(location="test2") |
2419 | + self.mock_with_return( |
2420 | + "snapstore_library.authentication.login", |
2421 | + (root_macaroon, discharge_macaroon), |
2422 | + ) |
2423 | + self.mock_with_return("getpass.getpass", "secret") |
2424 | |
2425 | - @responses.activate |
2426 | - def test_login_sso_unauthorized(self): |
2427 | - self.mock_input.return_value = 'user@example.org' |
2428 | - self.mock_getpass.return_value = 'secret' |
2429 | - self.add_issue_store_admin_response( |
2430 | - {'status': 200, 'json': {'macaroon': self.make_root_macaroon()}}) |
2431 | - auth_error = {'message': 'Provided email/password is not correct.'} |
2432 | - self.add_get_sso_discharge_response( |
2433 | - {'status': 401, 'json': {'error_list': [auth_error]}}) |
2434 | - self.assertEqual(1, login(self.make_args())) |
2435 | - self.assertEqual( |
2436 | - 'Enter your Ubuntu One SSO credentials.\n' |
2437 | - 'Login failed.\n' |
2438 | - 'Authentication error: Provided email/password is not correct.\n', |
2439 | - self.logger.output) |
2440 | + exit_status = self.run_main( |
2441 | + "login", "http://store.local/", "test@example.com" |
2442 | + ) |
2443 | |
2444 | - @responses.activate |
2445 | - def test_login_twofactor_required(self): |
2446 | - self.mock_input.side_effect = ('user@example.org', '123456') |
2447 | - self.mock_getpass.return_value = 'secret' |
2448 | - root = self.make_root_macaroon() |
2449 | - self.add_issue_store_admin_response( |
2450 | - {'status': 200, 'json': {'macaroon': root}}) |
2451 | - self.add_get_sso_discharge_response( |
2452 | - {'status': 401, |
2453 | - 'json': {'error_list': [{'code': 'twofactor-required'}]}}, |
2454 | - {'status': 200, 'json': {'discharge_macaroon': 'dummy'}}) |
2455 | - self.assertEqual(0, login(self.make_args())) |
2456 | + assert_that(exit_status, Equals(0)) |
2457 | + assert_that( |
2458 | + self.logger.output, |
2459 | + Equals( |
2460 | + "Enter your Ubuntu One SSO credentials.\n" |
2461 | + "Login successful\n" |
2462 | + "Configured http://store.local/ as the default store\n" |
2463 | + ), |
2464 | + ) |
2465 | |
2466 | - self.assertIn( |
2467 | - 'Enter your Ubuntu One SSO credentials.', self.logger.output) |
2468 | - self.mock_input.assert_has_calls([ |
2469 | - mock.call('Email: '), mock.call('Second-factor auth: ')]) |
2470 | - self.mock_getpass.assert_called_once_with('Password: ') |
2471 | - self.assertEqual(3, len(responses.calls)) |
2472 | - self.assertEqual({ |
2473 | - 'email': 'user@example.org', |
2474 | - 'password': 'secret', |
2475 | - 'caveat_id': 'payload', |
2476 | - }, json.loads(responses.calls[1].request.body.decode())) |
2477 | - self.assertEqual({ |
2478 | - 'email': 'user@example.org', |
2479 | - 'password': 'secret', |
2480 | - 'caveat_id': 'payload', |
2481 | - 'otp': '123456', |
2482 | - }, json.loads(responses.calls[2].request.body.decode())) |
2483 | - self.assertThat(config.Config().parser, ContainsDict({ |
2484 | - 'store:default': MatchesDict({ |
2485 | - 'gw_url': Equals(self.default_gw_url), |
2486 | - 'sso_url': Equals(self.default_sso_url), |
2487 | - 'root': Equals(root), |
2488 | - 'unbound_discharge': Equals('dummy'), |
2489 | - 'email': Equals('user@example.org'), |
2490 | - }), |
2491 | - })) |
2492 | + def test_login_two_factor_required(self): |
2493 | + root_macaroon = pymacaroons.Macaroon(location="test1") |
2494 | + discharge_macaroon = pymacaroons.Macaroon(location="test2") |
2495 | |
2496 | - @responses.activate |
2497 | - def test_login_twofactor_not_required(self): |
2498 | - self.mock_input.return_value = 'user@example.org' |
2499 | - self.mock_getpass.return_value = 'secret' |
2500 | - root = self.make_root_macaroon() |
2501 | - self.add_issue_store_admin_response( |
2502 | - {'status': 200, 'json': {'macaroon': root}}) |
2503 | - self.add_get_sso_discharge_response( |
2504 | - {'status': 200, 'json': {'discharge_macaroon': 'dummy'}}) |
2505 | - login(self.make_args()) |
2506 | + def _login(session, auth_url, sso_url, email, password, otp=None): |
2507 | + if otp is None: |
2508 | + raise exceptions.TwoFactorAuthenticationRequired() |
2509 | + assert_that(otp, Equals("123456")) |
2510 | + return root_macaroon, discharge_macaroon |
2511 | |
2512 | - self.assertIn( |
2513 | - 'Enter your Ubuntu One SSO credentials.', self.logger.output) |
2514 | - self.mock_input.assert_called_once_with('Email: ') |
2515 | - self.mock_getpass.assert_called_once_with('Password: ') |
2516 | - self.assertEqual(2, len(responses.calls)) |
2517 | - self.assertEqual({ |
2518 | - 'email': 'user@example.org', |
2519 | - 'password': 'secret', |
2520 | - 'caveat_id': 'payload', |
2521 | - }, json.loads(responses.calls[1].request.body.decode())) |
2522 | - self.assertThat(config.Config().parser, ContainsDict({ |
2523 | - 'store:default': MatchesDict({ |
2524 | - 'gw_url': Equals(self.default_gw_url), |
2525 | - 'sso_url': Equals(self.default_sso_url), |
2526 | - 'root': Equals(root), |
2527 | - 'unbound_discharge': Equals('dummy'), |
2528 | - 'email': Equals('user@example.org'), |
2529 | - }), |
2530 | - })) |
2531 | + self.mock_with_side_effect( |
2532 | + "snapstore_library.authentication.login", _login |
2533 | + ) |
2534 | + self.mock_with_return("getpass.getpass", "secret") |
2535 | + self.mock_with_return("builtins.input", "123456") |
2536 | |
2537 | - @responses.activate |
2538 | - def test_login_with_email(self): |
2539 | - self.mock_input.side_effect = Exception("shouldn't be called") |
2540 | - self.mock_getpass.return_value = 'secret' |
2541 | - root = self.make_root_macaroon() |
2542 | - self.add_issue_store_admin_response( |
2543 | - {'status': 200, 'json': {'macaroon': root}}) |
2544 | - self.add_get_sso_discharge_response( |
2545 | - {'status': 200, 'json': {'discharge_macaroon': 'dummy'}}) |
2546 | - login(self.make_args(email='user@example.org')) |
2547 | + exit_status = self.run_main( |
2548 | + "login", "http://store.local/", "test@example.com" |
2549 | + ) |
2550 | |
2551 | - self.assertIn( |
2552 | - 'Enter your Ubuntu One SSO credentials.', self.logger.output) |
2553 | - self.mock_input.assert_not_called() |
2554 | - self.mock_getpass.assert_called_once_with('Password: ') |
2555 | - self.assertEqual(2, len(responses.calls)) |
2556 | - self.assertEqual({ |
2557 | - 'email': 'user@example.org', |
2558 | - 'password': 'secret', |
2559 | - 'caveat_id': 'payload', |
2560 | - }, json.loads(responses.calls[1].request.body.decode())) |
2561 | - self.assertThat(config.Config().parser, ContainsDict({ |
2562 | - 'store:default': MatchesDict({ |
2563 | - 'gw_url': Equals(self.default_gw_url), |
2564 | - 'sso_url': Equals(self.default_sso_url), |
2565 | - 'root': Equals(root), |
2566 | - 'unbound_discharge': Equals('dummy'), |
2567 | - 'email': Equals('user@example.org'), |
2568 | - }), |
2569 | - })) |
2570 | + assert_that(exit_status, Equals(0)) |
2571 | + assert_that( |
2572 | + self.logger.output, |
2573 | + Equals( |
2574 | + "Enter your Ubuntu One SSO credentials.\n" |
2575 | + "Login successful\n" |
2576 | + "Configured http://store.local/ as the default store\n" |
2577 | + ), |
2578 | + ) |
2579 | |
2580 | - @responses.activate |
2581 | - def test_store_url(self): |
2582 | - gw_url = 'http://otherstore.local:1234/' |
2583 | + def test_login_no_email_arg(self): |
2584 | + root_macaroon = pymacaroons.Macaroon(location="test1") |
2585 | + discharge_macaroon = pymacaroons.Macaroon(location="test2") |
2586 | |
2587 | - self.mock_input.return_value = 'user@example.org' |
2588 | - self.mock_getpass.return_value = 'secret' |
2589 | - root = self.make_root_macaroon() |
2590 | - self.add_issue_store_admin_response( |
2591 | - {'status': 200, 'json': {'macaroon': root}}, gw_url=gw_url) |
2592 | - self.add_get_sso_discharge_response( |
2593 | - {'status': 200, 'json': {'discharge_macaroon': 'dummy'}}) |
2594 | - login(self.make_args(store_url=gw_url)) |
2595 | + def _login(session, auth_url, sso_url, email, password, otp=None): |
2596 | + assert_that(email, Equals("test@example.com")) |
2597 | + return root_macaroon, discharge_macaroon |
2598 | |
2599 | - self.assertEqual(2, len(responses.calls)) |
2600 | - self.assertEqual(responses.calls[0].request.url[:len(gw_url)], gw_url) |
2601 | - self.assertTrue( |
2602 | - responses.calls[1].request.url.startswith(self.default_sso_url)) |
2603 | - self.assertThat(config.Config().parser, ContainsDict({ |
2604 | - 'store:default': ContainsDict({ |
2605 | - 'gw_url': Equals(gw_url), |
2606 | - 'sso_url': Equals(self.default_sso_url), |
2607 | - }), |
2608 | - })) |
2609 | + self.mock_with_side_effect( |
2610 | + "snapstore_library.authentication.login", _login |
2611 | + ) |
2612 | + self.mock_with_return("builtins.input", "test@example.com") |
2613 | + self.mock_with_return("getpass.getpass", "secret") |
2614 | + |
2615 | + exit_status = self.run_main("login", "http://store.local/") |
2616 | + |
2617 | + assert_that(exit_status, Equals(0)) |
2618 | + assert_that( |
2619 | + self.logger.output, |
2620 | + Equals( |
2621 | + "Enter your Ubuntu One SSO credentials.\n" |
2622 | + "Login successful\n" |
2623 | + "Configured http://store.local/ as the default store\n" |
2624 | + ), |
2625 | + ) |
2626 | |
2627 | - @responses.activate |
2628 | - def test_sso_url(self): |
2629 | - sso_url = 'https://othersso.local:1234/' |
2630 | + def test_login_unhandled_error(self): |
2631 | + self.mock_with_side_effect( |
2632 | + "snapstore_library.authentication.login", |
2633 | + Exception("Some error happened"), |
2634 | + ) |
2635 | + self.mock_with_return("getpass.getpass", "secret") |
2636 | |
2637 | - self.mock_input.return_value = 'user@example.org' |
2638 | - self.mock_getpass.return_value = 'secret' |
2639 | - root = self.make_root_macaroon(sso_url=sso_url) |
2640 | - self.add_issue_store_admin_response( |
2641 | - {'status': 200, 'json': {'macaroon': root}}) |
2642 | - self.add_get_sso_discharge_response( |
2643 | - {'status': 200, 'json': {'discharge_macaroon': 'dummy'}}, |
2644 | - sso_url=sso_url) |
2645 | - login(self.make_args(sso_url=sso_url)) |
2646 | + exit_status = self.run_main( |
2647 | + "login", "http://store.local/", "test@example.com" |
2648 | + ) |
2649 | |
2650 | - self.assertEqual(2, len(responses.calls)) |
2651 | - self.assertTrue( |
2652 | - responses.calls[0].request.url.startswith(self.default_gw_url)) |
2653 | - self.assertEqual( |
2654 | - responses.calls[1].request.url[:len(sso_url)], sso_url) |
2655 | - self.assertThat(config.Config().parser, ContainsDict({ |
2656 | - 'store:default': ContainsDict({ |
2657 | - 'gw_url': Equals(self.default_gw_url), |
2658 | - 'sso_url': Equals(sso_url), |
2659 | - }), |
2660 | - })) |
2661 | + assert_that(exit_status, Equals(1)) |
2662 | + assert_that( |
2663 | + self.logger.output, |
2664 | + Equals( |
2665 | + "Enter your Ubuntu One SSO credentials.\n" |
2666 | + "Exception: Some error happened\n" |
2667 | + ), |
2668 | + ) |
2669 | diff --git a/snapstore_client/logic/tests/test_overrides.py b/snapstore_client/logic/tests/test_overrides.py |
2670 | index 05e32fc..5d53b48 100644 |
2671 | --- a/snapstore_client/logic/tests/test_overrides.py |
2672 | +++ b/snapstore_client/logic/tests/test_overrides.py |
2673 | @@ -1,314 +1,107 @@ |
2674 | # Copyright 2017 Canonical Ltd. |
2675 | - |
2676 | -import json |
2677 | -from urllib.parse import urljoin |
2678 | - |
2679 | -import fixtures |
2680 | -import responses |
2681 | -from testtools import TestCase |
2682 | - |
2683 | -from snapstore_client import config |
2684 | -from snapstore_client.logic.overrides import ( |
2685 | - delete_override, |
2686 | - list_overrides, |
2687 | - override, |
2688 | -) |
2689 | -from snapstore_client.tests import ( |
2690 | - factory, |
2691 | - testfixtures, |
2692 | -) |
2693 | - |
2694 | - |
2695 | -class OverridesTests(TestCase): |
2696 | - |
2697 | - def test_list_overrides_no_store_config(self): |
2698 | - self.useFixture(testfixtures.ConfigFixture(empty=True)) |
2699 | - logger = self.useFixture(fixtures.FakeLogger()) |
2700 | - rc = list_overrides(factory.Args(snap_name='some-snap', series='16')) |
2701 | - self.assertEqual(rc, 1) |
2702 | - self.assertEqual( |
2703 | - logger.output, |
2704 | - 'No store configuration found. ' |
2705 | - 'Have you run "snap-store-proxy-client login"?\n') |
2706 | - |
2707 | - @responses.activate |
2708 | - def test_list_overrides_online(self): |
2709 | - self.useFixture(testfixtures.ConfigFixture()) |
2710 | - logger = self.useFixture(fixtures.FakeLogger()) |
2711 | - snap_id = factory.generate_snap_id() |
2712 | - overrides = [ |
2713 | - factory.SnapDeviceGateway.Override( |
2714 | - snap_id=snap_id, snap_name='mysnap'), |
2715 | - factory.SnapDeviceGateway.Override( |
2716 | - snap_id=snap_id, snap_name='mysnap', revision=3, |
2717 | - upstream_revision=4, channel='foo/stable', |
2718 | - architecture='i386'), |
2719 | - ] |
2720 | - # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once |
2721 | - # they exist. |
2722 | - overrides_url = urljoin( |
2723 | - config.Config().store_section('default').get('gw_url'), |
2724 | - '/v2/metadata/overrides/mysnap') |
2725 | - responses.add( |
2726 | - 'GET', overrides_url, status=200, json={'overrides': overrides}) |
2727 | - |
2728 | - list_overrides( |
2729 | - factory.Args(snap_name='mysnap', series='16', password=False)) |
2730 | - self.assertEqual( |
2731 | - 'mysnap stable amd64 1 (upstream 2)\n' |
2732 | - 'mysnap foo/stable i386 3 (upstream 4)\n', |
2733 | - logger.output) |
2734 | - # We shouldn't have Basic Authorization headers, but Macaroon |
2735 | - self.assertNotIn( |
2736 | - 'Basic', |
2737 | - responses.calls[0].request.headers['Authorization']) |
2738 | - |
2739 | - @responses.activate |
2740 | - def test_list_overrides_offline(self): |
2741 | - self.useFixture(testfixtures.ConfigFixture()) |
2742 | - logger = self.useFixture(fixtures.FakeLogger()) |
2743 | - snap_id = factory.generate_snap_id() |
2744 | - overrides = [ |
2745 | - factory.SnapDeviceGateway.Override( |
2746 | - snap_id=snap_id, snap_name='mysnap'), |
2747 | - factory.SnapDeviceGateway.Override( |
2748 | - snap_id=snap_id, snap_name='mysnap', revision=3, |
2749 | - upstream_revision=4, channel='foo/stable', |
2750 | - architecture='i386'), |
2751 | - ] |
2752 | - # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once |
2753 | - # they exist. |
2754 | - overrides_url = urljoin( |
2755 | - config.Config().store_section('default').get('gw_url'), |
2756 | - '/v2/metadata/overrides/mysnap') |
2757 | - responses.add( |
2758 | - 'GET', overrides_url, status=200, json={'overrides': overrides}) |
2759 | - |
2760 | - list_overrides( |
2761 | - factory.Args(snap_name='mysnap', series='16', password='test')) |
2762 | - self.assertEqual( |
2763 | - 'mysnap stable amd64 1 (upstream 2)\n' |
2764 | - 'mysnap foo/stable i386 3 (upstream 4)\n', |
2765 | - logger.output) |
2766 | - self.assertEqual( |
2767 | - 'Basic YWRtaW46dGVzdA==', |
2768 | - responses.calls[0].request.headers['Authorization']) |
2769 | - |
2770 | - def test_override_no_store_config(self): |
2771 | - self.useFixture(testfixtures.ConfigFixture(empty=True)) |
2772 | - logger = self.useFixture(fixtures.FakeLogger()) |
2773 | - rc = override(factory.Args( |
2774 | - snap_name='some-snap', channel_map_entries=['stable=1'], |
2775 | - series='16', |
2776 | - password=False)) |
2777 | - self.assertEqual(rc, 1) |
2778 | - self.assertEqual( |
2779 | - logger.output, |
2780 | - 'No store configuration found. ' |
2781 | - 'Have you run "snap-store-proxy-client login"?\n') |
2782 | - |
2783 | - @responses.activate |
2784 | - def test_override_online(self): |
2785 | - self.useFixture(testfixtures.ConfigFixture()) |
2786 | - logger = self.useFixture(fixtures.FakeLogger()) |
2787 | - snap_id = factory.generate_snap_id() |
2788 | - overrides = [ |
2789 | - factory.SnapDeviceGateway.Override( |
2790 | - snap_id=snap_id, snap_name='mysnap'), |
2791 | - factory.SnapDeviceGateway.Override( |
2792 | - snap_id=snap_id, snap_name='mysnap', revision=3, |
2793 | - upstream_revision=4, channel='foo/stable', |
2794 | - architecture='i386'), |
2795 | - ] |
2796 | - # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once |
2797 | - # they exist. |
2798 | - overrides_url = urljoin( |
2799 | - config.Config().store_section('default').get('gw_url'), |
2800 | - '/v2/metadata/overrides') |
2801 | - responses.add( |
2802 | - 'POST', overrides_url, status=200, json={'overrides': overrides}) |
2803 | - |
2804 | - override(factory.Args( |
2805 | - snap_name='mysnap', |
2806 | - channel_map_entries=['stable=1', 'foo/stable=3'], |
2807 | - series='16', |
2808 | - password=False)) |
2809 | - self.assertEqual([ |
2810 | - { |
2811 | - 'snap_name': 'mysnap', |
2812 | - 'revision': 1, |
2813 | - 'channel': 'stable', |
2814 | - 'series': '16', |
2815 | - }, |
2816 | - { |
2817 | - 'snap_name': 'mysnap', |
2818 | - 'revision': 3, |
2819 | - 'channel': 'foo/stable', |
2820 | - 'series': '16', |
2821 | - }, |
2822 | - ], json.loads(responses.calls[0].request.body.decode())) |
2823 | - self.assertEqual( |
2824 | - 'mysnap stable amd64 1 (upstream 2)\n' |
2825 | - 'mysnap foo/stable i386 3 (upstream 4)\n', |
2826 | - logger.output) |
2827 | - # We shouldn't have Basic Authorization headers, but Macaroon |
2828 | - self.assertNotIn( |
2829 | - 'Basic', |
2830 | - responses.calls[0].request.headers['Authorization']) |
2831 | - |
2832 | - @responses.activate |
2833 | - def test_override_offline(self): |
2834 | - self.useFixture(testfixtures.ConfigFixture()) |
2835 | - logger = self.useFixture(fixtures.FakeLogger()) |
2836 | - snap_id = factory.generate_snap_id() |
2837 | - overrides = [ |
2838 | - factory.SnapDeviceGateway.Override( |
2839 | - snap_id=snap_id, snap_name='mysnap'), |
2840 | - factory.SnapDeviceGateway.Override( |
2841 | - snap_id=snap_id, snap_name='mysnap', revision=3, |
2842 | - upstream_revision=4, channel='foo/stable', |
2843 | - architecture='i386'), |
2844 | - ] |
2845 | - # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once |
2846 | - # they exist. |
2847 | - overrides_url = urljoin( |
2848 | - config.Config().store_section('default').get('gw_url'), |
2849 | - '/v2/metadata/overrides') |
2850 | - responses.add( |
2851 | - 'POST', overrides_url, status=200, json={'overrides': overrides}) |
2852 | - |
2853 | - override(factory.Args( |
2854 | - snap_name='mysnap', |
2855 | - channel_map_entries=['stable=1', 'foo/stable=3'], |
2856 | - series='16', |
2857 | - password='test')) |
2858 | - self.assertEqual([ |
2859 | - { |
2860 | - 'snap_name': 'mysnap', |
2861 | - 'revision': 1, |
2862 | - 'channel': 'stable', |
2863 | - 'series': '16', |
2864 | - }, |
2865 | - { |
2866 | - 'snap_name': 'mysnap', |
2867 | - 'revision': 3, |
2868 | - 'channel': 'foo/stable', |
2869 | - 'series': '16', |
2870 | - }, |
2871 | - ], json.loads(responses.calls[0].request.body.decode())) |
2872 | - self.assertEqual( |
2873 | - 'mysnap stable amd64 1 (upstream 2)\n' |
2874 | - 'mysnap foo/stable i386 3 (upstream 4)\n', |
2875 | - logger.output) |
2876 | - self.assertEqual( |
2877 | - 'Basic YWRtaW46dGVzdA==', |
2878 | - responses.calls[0].request.headers['Authorization']) |
2879 | - |
2880 | - def test_delete_override_no_store_config(self): |
2881 | - self.useFixture(testfixtures.ConfigFixture(empty=True)) |
2882 | - logger = self.useFixture(fixtures.FakeLogger()) |
2883 | - rc = delete_override(factory.Args( |
2884 | - snap_name='some-snap', channels=['stable'], |
2885 | - series='16', password=False)) |
2886 | - self.assertEqual(rc, 1) |
2887 | - self.assertEqual( |
2888 | - logger.output, |
2889 | - 'No store configuration found. ' |
2890 | - 'Have you run "snap-store-proxy-client login"?\n') |
2891 | - |
2892 | - @responses.activate |
2893 | - def test_delete_override_online(self): |
2894 | - self.useFixture(testfixtures.ConfigFixture()) |
2895 | - logger = self.useFixture(fixtures.FakeLogger()) |
2896 | - snap_id = factory.generate_snap_id() |
2897 | - overrides = [ |
2898 | - factory.SnapDeviceGateway.Override( |
2899 | - snap_id=snap_id, snap_name='mysnap', revision=None), |
2900 | - factory.SnapDeviceGateway.Override( |
2901 | - snap_id=snap_id, snap_name='mysnap', revision=None, |
2902 | - upstream_revision=4, channel='foo/stable', |
2903 | - architecture='i386'), |
2904 | - ] |
2905 | - # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once |
2906 | - # they exist. |
2907 | - overrides_url = urljoin( |
2908 | - config.Config().store_section('default').get('gw_url'), |
2909 | - '/v2/metadata/overrides') |
2910 | - responses.add( |
2911 | - 'POST', overrides_url, status=200, json={'overrides': overrides}) |
2912 | - |
2913 | - delete_override(factory.Args( |
2914 | - snap_name='mysnap', |
2915 | - channels=['stable', 'foo/stable'], |
2916 | - series='16', |
2917 | - password=False)) |
2918 | - self.assertEqual([ |
2919 | - { |
2920 | - 'snap_name': 'mysnap', |
2921 | - 'revision': None, |
2922 | - 'channel': 'stable', |
2923 | - 'series': '16', |
2924 | - }, |
2925 | - { |
2926 | - 'snap_name': 'mysnap', |
2927 | - 'revision': None, |
2928 | - 'channel': 'foo/stable', |
2929 | - 'series': '16', |
2930 | - }, |
2931 | - ], json.loads(responses.calls[0].request.body.decode())) |
2932 | - self.assertEqual( |
2933 | - 'mysnap stable amd64 is tracking upstream (revision 2)\n' |
2934 | - 'mysnap foo/stable i386 is tracking upstream (revision 4)\n', |
2935 | - logger.output) |
2936 | - # We shouldn't have Basic Authorization headers, but Macaroon |
2937 | - self.assertNotIn( |
2938 | - 'Basic', |
2939 | - responses.calls[0].request.headers['Authorization']) |
2940 | - |
2941 | - @responses.activate |
2942 | - def test_delete_override_offline(self): |
2943 | - self.useFixture(testfixtures.ConfigFixture()) |
2944 | - logger = self.useFixture(fixtures.FakeLogger()) |
2945 | - snap_id = factory.generate_snap_id() |
2946 | - overrides = [ |
2947 | - factory.SnapDeviceGateway.Override( |
2948 | - snap_id=snap_id, snap_name='mysnap', revision=None), |
2949 | - factory.SnapDeviceGateway.Override( |
2950 | - snap_id=snap_id, snap_name='mysnap', revision=None, |
2951 | - upstream_revision=4, channel='foo/stable', |
2952 | - architecture='i386'), |
2953 | - ] |
2954 | - # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once |
2955 | - # they exist. |
2956 | - overrides_url = urljoin( |
2957 | - config.Config().store_section('default').get('gw_url'), |
2958 | - '/v2/metadata/overrides') |
2959 | - responses.add( |
2960 | - 'POST', overrides_url, status=200, json={'overrides': overrides}) |
2961 | - |
2962 | - delete_override(factory.Args( |
2963 | - snap_name='mysnap', |
2964 | - channels=['stable', 'foo/stable'], |
2965 | - series='16', |
2966 | - password='test')) |
2967 | - self.assertEqual([ |
2968 | - { |
2969 | - 'snap_name': 'mysnap', |
2970 | - 'revision': None, |
2971 | - 'channel': 'stable', |
2972 | - 'series': '16', |
2973 | - }, |
2974 | - { |
2975 | - 'snap_name': 'mysnap', |
2976 | - 'revision': None, |
2977 | - 'channel': 'foo/stable', |
2978 | - 'series': '16', |
2979 | - }, |
2980 | - ], json.loads(responses.calls[0].request.body.decode())) |
2981 | - self.assertEqual( |
2982 | - 'mysnap stable amd64 is tracking upstream (revision 2)\n' |
2983 | - 'mysnap foo/stable i386 is tracking upstream (revision 4)\n', |
2984 | - logger.output) |
2985 | - self.assertEqual( |
2986 | - 'Basic YWRtaW46dGVzdA==', |
2987 | - responses.calls[0].request.headers['Authorization']) |
2988 | +from testtools.assertions import assert_that |
2989 | +from testtools.matchers import Equals |
2990 | + |
2991 | +from snapstore_client.logic.tests.utils import RunMainTestCase |
2992 | + |
2993 | + |
2994 | +class OverridesTests(RunMainTestCase): |
2995 | + def test_list_overrides(self): |
2996 | + overrides = { |
2997 | + "overrides": [ |
2998 | + { |
2999 | + "snap_id": "x" * 32, |
3000 | + "snap_name": "mysnap", |
3001 | + "revision": 1, |
3002 | + "upstream_revision": 2, |
3003 | + "channel": "stable", |
3004 | + "architecture": "amd64", |
3005 | + "series": 16, |
3006 | + }, |
3007 | + { |
3008 | + "snap_id": "x" * 32, |
3009 | + "snap_name": "mysnap", |
3010 | + "revision": 3, |
3011 | + "upstream_revision": 5, |
3012 | + "channel": "foo/stable", |
3013 | + "architecture": "i386", |
3014 | + "series": 16, |
3015 | + }, |
3016 | + ] |
3017 | + } |
3018 | + self.mock_with_return( |
3019 | + "snapstore_library.overrides.get_overrides", overrides |
3020 | + ) |
3021 | + |
3022 | + self.run_main_preconfigured("list-overrides", "mysnap") |
3023 | + |
3024 | + assert_that( |
3025 | + self.logger.output, |
3026 | + Equals( |
3027 | + "mysnap stable amd64 1 (upstream 2)\n" |
3028 | + "mysnap foo/stable i386 3 (upstream 5)\n" |
3029 | + ), |
3030 | + ) |
3031 | + |
3032 | + def test_overrides(self): |
3033 | + overrides = { |
3034 | + "overrides": [ |
3035 | + { |
3036 | + "snap_id": "x" * 32, |
3037 | + "snap_name": "mysnap", |
3038 | + "revision": 1, |
3039 | + "upstream_revision": 6, |
3040 | + "channel": "stable", |
3041 | + "architecture": "amd64", |
3042 | + "series": 16, |
3043 | + }, |
3044 | + { |
3045 | + "snap_id": "x" * 32, |
3046 | + "snap_name": "mysnap", |
3047 | + "revision": 3, |
3048 | + "upstream_revision": 7, |
3049 | + "channel": "foo/stable", |
3050 | + "architecture": "i386", |
3051 | + "series": 16, |
3052 | + }, |
3053 | + ] |
3054 | + } |
3055 | + self.mock_with_return( |
3056 | + "snapstore_library.overrides.set_overrides", overrides |
3057 | + ) |
3058 | + |
3059 | + self.run_main_preconfigured( |
3060 | + "override", "mysnap", "foo/stable=3", "stable=1" |
3061 | + ) |
3062 | + |
3063 | + assert_that( |
3064 | + self.logger.output, |
3065 | + Equals( |
3066 | + "mysnap stable amd64 1 (upstream 6)\n" |
3067 | + "mysnap foo/stable i386 3 (upstream 7)\n" |
3068 | + ), |
3069 | + ) |
3070 | + |
3071 | + def test_delete_overrides(self): |
3072 | + overrides = { |
3073 | + "overrides": [ |
3074 | + { |
3075 | + "snap_id": "x" * 32, |
3076 | + "snap_name": "mysnap", |
3077 | + "revision": 1, |
3078 | + "upstream_revision": 6, |
3079 | + "channel": "stable", |
3080 | + "architecture": "amd64", |
3081 | + "series": 16, |
3082 | + } |
3083 | + ] |
3084 | + } |
3085 | + self.mock_with_return( |
3086 | + "snapstore_library.overrides.set_overrides", overrides |
3087 | + ) |
3088 | + |
3089 | + self.run_main_preconfigured("delete-override", "mysnap", "foo/stable") |
3090 | + |
3091 | + assert_that( |
3092 | + self.logger.output, Equals("mysnap stable amd64 1 (upstream 6)\n") |
3093 | + ) |
3094 | diff --git a/snapstore_client/logic/tests/utils.py b/snapstore_client/logic/tests/utils.py |
3095 | new file mode 100644 |
3096 | index 0000000..fd2d596 |
3097 | --- /dev/null |
3098 | +++ b/snapstore_client/logic/tests/utils.py |
3099 | @@ -0,0 +1,42 @@ |
3100 | +import sys |
3101 | + |
3102 | +import testtools |
3103 | +import fixtures |
3104 | + |
3105 | +from snapstore_client.__main__ import main |
3106 | +from snapstore_client.tests import testfixtures |
3107 | + |
3108 | + |
3109 | +class RunMainTestCase(testtools.TestCase): |
3110 | + def mock(self, function): |
3111 | + return self.useFixture(fixtures.MockPatch(function)).mock |
3112 | + |
3113 | + def mock_with_return(self, function, rtn): |
3114 | + mock = self.mock(function) |
3115 | + mock.return_value = rtn |
3116 | + return mock |
3117 | + |
3118 | + def mock_with_side_effect(self, function, side_effect): |
3119 | + mock = self.mock(function) |
3120 | + mock.side_effect = side_effect |
3121 | + return mock |
3122 | + |
3123 | + def _run_main(self, *args, empty_config=True): |
3124 | + self.useFixture(testfixtures.ConfigFixture(empty=empty_config)) |
3125 | + self.mock("snapstore_client.__main__.configure_logging") |
3126 | + self.logger = self.useFixture(fixtures.FakeLogger()) |
3127 | + |
3128 | + old_argv = sys.argv |
3129 | + try: |
3130 | + sys.argv = ["snap-store-proxy-client"] + list(args) |
3131 | + return main() |
3132 | + except SystemExit as se: |
3133 | + raise AssertionError("Exited with {}".format(se.code)) |
3134 | + finally: |
3135 | + sys.argv = old_argv |
3136 | + |
3137 | + def run_main(self, *args): |
3138 | + return self._run_main(*args, empty_config=True) |
3139 | + |
3140 | + def run_main_preconfigured(self, *args): |
3141 | + return self._run_main(*args, empty_config=False) |
3142 | diff --git a/snapstore_client/tests/test_webservices.py b/snapstore_client/tests/test_webservices.py |
3143 | deleted file mode 100644 |
3144 | index c9c743c..0000000 |
3145 | --- a/snapstore_client/tests/test_webservices.py |
3146 | +++ /dev/null |
3147 | @@ -1,228 +0,0 @@ |
3148 | -# Copyright 2017 Canonical Ltd. |
3149 | - |
3150 | -import json |
3151 | -import sys |
3152 | -from urllib.parse import urljoin |
3153 | - |
3154 | -import fixtures |
3155 | -from requests.exceptions import HTTPError |
3156 | -import responses |
3157 | -from testtools import TestCase |
3158 | - |
3159 | -from snapstore_client import ( |
3160 | - config, |
3161 | - exceptions, |
3162 | - webservices, |
3163 | -) |
3164 | -from snapstore_client.tests import ( |
3165 | - factory, |
3166 | - matchers, |
3167 | - testfixtures, |
3168 | -) |
3169 | - |
3170 | -if sys.version < '3.6': |
3171 | - import sha3 # noqa |
3172 | - |
3173 | - |
3174 | -class WebservicesTests(TestCase): |
3175 | - |
3176 | - def setUp(self): |
3177 | - super().setUp() |
3178 | - self.config = self.useFixture(testfixtures.ConfigFixture()) |
3179 | - |
3180 | - @responses.activate |
3181 | - def test_issue_store_admin_success(self): |
3182 | - gw_url = 'http://store.local/' |
3183 | - issue_store_admin_url = urljoin(gw_url, '/v2/auth/issue-store-admin') |
3184 | - responses.add( |
3185 | - 'POST', issue_store_admin_url, status=200, |
3186 | - json={'macaroon': 'dummy'}) |
3187 | - |
3188 | - self.assertEqual('dummy', webservices.issue_store_admin(gw_url)) |
3189 | - |
3190 | - @responses.activate |
3191 | - def test_issue_store_admin_error(self): |
3192 | - logger = self.useFixture(fixtures.FakeLogger()) |
3193 | - gw_url = 'http://store.local/' |
3194 | - issue_store_admin_url = urljoin(gw_url, '/v2/auth/issue-store-admin') |
3195 | - responses.add( |
3196 | - 'POST', issue_store_admin_url, status=400, |
3197 | - json=factory.APIError.single('Something went wrong').to_dict()) |
3198 | - |
3199 | - self.assertRaises( |
3200 | - HTTPError, webservices.issue_store_admin, gw_url) |
3201 | - self.assertEqual( |
3202 | - 'Failed to issue store_admin macaroon:\nSomething went wrong\n', |
3203 | - logger.output) |
3204 | - |
3205 | - @responses.activate |
3206 | - def test_get_sso_discharge_success(self): |
3207 | - sso_url = 'http://sso.local/' |
3208 | - discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge') |
3209 | - responses.add( |
3210 | - 'POST', discharge_url, status=200, |
3211 | - json={'discharge_macaroon': 'dummy'}) |
3212 | - |
3213 | - self.assertEqual( |
3214 | - 'dummy', |
3215 | - webservices.get_sso_discharge( |
3216 | - sso_url, 'user@example.org', 'secret', 'caveat')) |
3217 | - request = responses.calls[0].request |
3218 | - self.assertEqual('application/json', request.headers['Content-Type']) |
3219 | - self.assertEqual({ |
3220 | - 'email': 'user@example.org', |
3221 | - 'password': 'secret', |
3222 | - 'caveat_id': 'caveat', |
3223 | - }, json.loads(request.body.decode())) |
3224 | - |
3225 | - @responses.activate |
3226 | - def test_get_sso_discharge_success_with_otp(self): |
3227 | - sso_url = 'http://sso.local/' |
3228 | - discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge') |
3229 | - responses.add( |
3230 | - 'POST', discharge_url, status=200, |
3231 | - json={'discharge_macaroon': 'dummy'}) |
3232 | - |
3233 | - self.assertEqual( |
3234 | - 'dummy', |
3235 | - webservices.get_sso_discharge( |
3236 | - sso_url, 'user@example.org', 'secret', 'caveat', |
3237 | - one_time_password='123456')) |
3238 | - request = responses.calls[0].request |
3239 | - self.assertEqual('application/json', request.headers['Content-Type']) |
3240 | - self.assertEqual({ |
3241 | - 'email': 'user@example.org', |
3242 | - 'password': 'secret', |
3243 | - 'caveat_id': 'caveat', |
3244 | - 'otp': '123456', |
3245 | - }, json.loads(request.body.decode())) |
3246 | - |
3247 | - @responses.activate |
3248 | - def test_get_sso_discharge_twofactor_required(self): |
3249 | - sso_url = 'http://sso.local/' |
3250 | - discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge') |
3251 | - responses.add( |
3252 | - 'POST', discharge_url, status=401, |
3253 | - json={'error_list': [{'code': 'twofactor-required'}]}) |
3254 | - |
3255 | - self.assertRaises( |
3256 | - exceptions.StoreTwoFactorAuthenticationRequired, |
3257 | - webservices.get_sso_discharge, |
3258 | - sso_url, 'user@example.org', 'secret', 'caveat') |
3259 | - |
3260 | - @responses.activate |
3261 | - def test_get_sso_discharge_structured_error(self): |
3262 | - sso_url = 'http://sso.local/' |
3263 | - discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge') |
3264 | - responses.add( |
3265 | - 'POST', discharge_url, status=400, |
3266 | - json={'error_list': [{'code': 'invalid-request', |
3267 | - 'message': 'Something went wrong'}]}) |
3268 | - |
3269 | - e = self.assertRaises( |
3270 | - exceptions.StoreAuthenticationError, webservices.get_sso_discharge, |
3271 | - sso_url, 'user@example.org', 'secret', 'caveat') |
3272 | - self.assertEqual('Something went wrong', e.message) |
3273 | - |
3274 | - @responses.activate |
3275 | - def test_get_sso_discharge_unstructured_error(self): |
3276 | - logger = self.useFixture(fixtures.FakeLogger()) |
3277 | - sso_url = 'http://sso.local/' |
3278 | - discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge') |
3279 | - responses.add( |
3280 | - 'POST', discharge_url, status=503, body='Try again later.') |
3281 | - |
3282 | - self.assertRaises( |
3283 | - HTTPError, webservices.get_sso_discharge, |
3284 | - sso_url, 'user@example.org', 'secret', 'caveat') |
3285 | - self.assertEqual( |
3286 | - 'Failed to get SSO discharge:\n' |
3287 | - '====================\n' |
3288 | - 'Try again later.\n' |
3289 | - '====================\n', |
3290 | - logger.output) |
3291 | - |
3292 | - @responses.activate |
3293 | - def test_get_overrides_success(self): |
3294 | - logger = self.useFixture(fixtures.FakeLogger()) |
3295 | - overrides = [factory.SnapDeviceGateway.Override()] |
3296 | - # XXX cjwatson 2017-06-26: Use acceptable-generated double once it |
3297 | - # exists. |
3298 | - store = config.Config().store_section('default') |
3299 | - overrides_url = urljoin( |
3300 | - store.get('gw_url'), '/v2/metadata/overrides/mysnap') |
3301 | - responses.add('GET', overrides_url, status=200, json=overrides) |
3302 | - |
3303 | - self.assertEqual(overrides, webservices.get_overrides( |
3304 | - store, 'mysnap')) |
3305 | - request = responses.calls[0].request |
3306 | - self.assertThat( |
3307 | - request.headers['Authorization'], |
3308 | - matchers.MacaroonHeaderVerifies(self.config.key)) |
3309 | - self.assertNotIn('Failed to get overrides:', logger.output) |
3310 | - |
3311 | - @responses.activate |
3312 | - def test_get_overrides_error(self): |
3313 | - logger = self.useFixture(fixtures.FakeLogger()) |
3314 | - store = config.Config().store_section('default') |
3315 | - overrides_url = urljoin( |
3316 | - store.get('gw_url'), '/v2/metadata/overrides/mysnap') |
3317 | - responses.add( |
3318 | - 'GET', overrides_url, status=400, |
3319 | - json=factory.APIError.single('Something went wrong').to_dict()) |
3320 | - |
3321 | - self.assertRaises( |
3322 | - HTTPError, webservices.get_overrides, store, 'mysnap') |
3323 | - self.assertEqual( |
3324 | - 'Failed to get overrides:\nSomething went wrong\n', logger.output) |
3325 | - |
3326 | - @responses.activate |
3327 | - def test_set_overrides_success(self): |
3328 | - logger = self.useFixture(fixtures.FakeLogger()) |
3329 | - override = factory.SnapDeviceGateway.Override() |
3330 | - # XXX cjwatson 2017-06-26: Use acceptable-generated double once it |
3331 | - # exists. |
3332 | - store = config.Config().store_section('default') |
3333 | - overrides_url = urljoin(store.get('gw_url'), '/v2/metadata/overrides') |
3334 | - responses.add('POST', overrides_url, status=200, json=[override]) |
3335 | - |
3336 | - self.assertEqual([override], webservices.set_overrides( |
3337 | - store, [{ |
3338 | - 'snap_name': override['snap_name'], |
3339 | - 'revision': override['revision'], |
3340 | - 'channel': override['channel'], |
3341 | - 'series': override['series'], |
3342 | - }])) |
3343 | - request = responses.calls[0].request |
3344 | - self.assertThat( |
3345 | - request.headers['Authorization'], |
3346 | - matchers.MacaroonHeaderVerifies(self.config.key)) |
3347 | - self.assertEqual([{ |
3348 | - 'snap_name': override['snap_name'], |
3349 | - 'revision': override['revision'], |
3350 | - 'channel': override['channel'], |
3351 | - 'series': override['series'], |
3352 | - }], json.loads(request.body.decode())) |
3353 | - self.assertNotIn('Failed to set override:', logger.output) |
3354 | - |
3355 | - @responses.activate |
3356 | - def test_set_overrides_error(self): |
3357 | - logger = self.useFixture(fixtures.FakeLogger()) |
3358 | - override = factory.SnapDeviceGateway.Override() |
3359 | - # XXX cjwatson 2017-06-26: Use acceptable-generated double once it |
3360 | - # exists. |
3361 | - store = config.Config().store_section('default') |
3362 | - overrides_url = urljoin(store.get('gw_url'), '/v2/metadata/overrides') |
3363 | - responses.add( |
3364 | - 'POST', overrides_url, status=400, |
3365 | - json=factory.APIError.single('Something went wrong').to_dict()) |
3366 | - |
3367 | - self.assertRaises(HTTPError, lambda: webservices.set_overrides( |
3368 | - store, { |
3369 | - 'snap_name': override['snap_name'], |
3370 | - 'revision': override['revision'], |
3371 | - 'channel': override['channel'], |
3372 | - 'series': override['series'], |
3373 | - })) |
3374 | - self.assertEqual( |
3375 | - 'Failed to set override:\nSomething went wrong\n', logger.output) |
3376 | diff --git a/snapstore_client/utils.py b/snapstore_client/utils.py |
3377 | index 461a1c3..8b10606 100644 |
3378 | --- a/snapstore_client/utils.py |
3379 | +++ b/snapstore_client/utils.py |
3380 | @@ -3,11 +3,6 @@ import logging |
3381 | logger = logging.getLogger(__name__) |
3382 | |
3383 | |
3384 | -def _log_credentials_error(e): |
3385 | - logger.error('%s', e) |
3386 | - logger.error('Try to "snap-store-proxy-client login" again.') |
3387 | - |
3388 | - |
3389 | def _log_authorized_error(): |
3390 | logger.error(("Perhaps you have not been registered as an " |
3391 | "admin with the proxy.")) |
3392 | diff --git a/snapstore_client/webservices.py b/snapstore_client/webservices.py |
3393 | deleted file mode 100644 |
3394 | index 3ebf149..0000000 |
3395 | --- a/snapstore_client/webservices.py |
3396 | +++ /dev/null |
3397 | @@ -1,185 +0,0 @@ |
3398 | -# Copyright 2017 Canonical Ltd. This software is licensed under the |
3399 | -# GNU General Public License version 3 (see the file LICENSE). |
3400 | - |
3401 | -import base64 |
3402 | -import json |
3403 | -import logging |
3404 | -import urllib.parse |
3405 | - |
3406 | -from pymacaroons import Macaroon |
3407 | -import requests |
3408 | - |
3409 | -from snapstore_client import exceptions |
3410 | - |
3411 | - |
3412 | -logger = logging.getLogger(__name__) |
3413 | - |
3414 | - |
3415 | -def issue_store_admin(gw_url): |
3416 | - """Ask the store to issue a store_admin macaroon.""" |
3417 | - issue_store_admin_url = urllib.parse.urljoin( |
3418 | - gw_url, '/v2/auth/issue-store-admin') |
3419 | - try: |
3420 | - resp = requests.post(issue_store_admin_url) |
3421 | - except requests.exceptions.ConnectionError: |
3422 | - raise exceptions.StoreCommunicationError(gw_url) |
3423 | - except requests.exceptions.RequestException: |
3424 | - raise exceptions.InvalidStoreURL(gw_url) |
3425 | - if resp.status_code == 404: |
3426 | - _print_error_message('issue store_admin macaroon', resp) |
3427 | - raise exceptions.InvalidStoreURL(gw_url) |
3428 | - elif resp.status_code != 200: |
3429 | - _print_error_message('issue store_admin macaroon', resp) |
3430 | - resp.raise_for_status() |
3431 | - return resp.json()['macaroon'] |
3432 | - |
3433 | - |
3434 | -def get_sso_discharge(sso_url, email, password, caveat_id, |
3435 | - one_time_password=None): |
3436 | - discharge_url = urllib.parse.urljoin( |
3437 | - sso_url, '/api/v2/tokens/discharge') |
3438 | - data = {'email': email, 'password': password, 'caveat_id': caveat_id} |
3439 | - if one_time_password is not None: |
3440 | - data['otp'] = one_time_password |
3441 | - resp = requests.post( |
3442 | - discharge_url, headers={'Accept': 'application/json'}, json=data) |
3443 | - if not resp.ok: |
3444 | - try: |
3445 | - error_list = resp.json().get('error_list', []) |
3446 | - if resp.status_code == 401: |
3447 | - for error in error_list: |
3448 | - if error.get('code') == 'twofactor-required': |
3449 | - raise exceptions.StoreTwoFactorAuthenticationRequired() |
3450 | - if error_list: |
3451 | - # Only bother about the first error. |
3452 | - error = error_list[0] |
3453 | - raise exceptions.StoreAuthenticationError( |
3454 | - error['message'], extra=error.get('extra')) |
3455 | - except json.JSONDecodeError: |
3456 | - pass |
3457 | - _print_error_message('get SSO discharge', resp) |
3458 | - resp.raise_for_status() |
3459 | - return resp.json()['discharge_macaroon'] |
3460 | - |
3461 | - |
3462 | -def refresh_sso_discharge(store, unbound_discharge_raw): |
3463 | - refresh_url = urllib.parse.urljoin( |
3464 | - store.get('sso_url'), '/api/v2/tokens/refresh') |
3465 | - data = {'discharge_macaroon': unbound_discharge_raw} |
3466 | - resp = requests.post( |
3467 | - refresh_url, headers={'Accept': 'application/json'}, json=data) |
3468 | - if not resp.ok: |
3469 | - _print_error_message('refresh SSO discharge', resp) |
3470 | - resp.raise_for_status() |
3471 | - return resp.json()['discharge_macaroon'] |
3472 | - |
3473 | - |
3474 | -def _deserialize_macaroon(name, value): |
3475 | - if value is None: |
3476 | - raise exceptions.InvalidCredentials('no {} macaroon'.format(name)) |
3477 | - try: |
3478 | - return Macaroon.deserialize(value) |
3479 | - except Exception: |
3480 | - raise exceptions.InvalidCredentials( |
3481 | - 'failed to deserialize {} macaroon'.format(name)) |
3482 | - |
3483 | - |
3484 | -def _get_macaroon_auth(store): |
3485 | - """Return an Authorization header containing store macaroons.""" |
3486 | - root_raw = store.get('root') |
3487 | - root = _deserialize_macaroon('root', root_raw) |
3488 | - unbound_discharge_raw = store.get('unbound_discharge') |
3489 | - unbound_discharge = _deserialize_macaroon( |
3490 | - 'unbound discharge', unbound_discharge_raw) |
3491 | - bound_discharge = root.prepare_for_request(unbound_discharge) |
3492 | - bound_discharge_raw = bound_discharge.serialize() |
3493 | - return 'Macaroon root="{}", discharge="{}"'.format( |
3494 | - root_raw, bound_discharge_raw) |
3495 | - |
3496 | - |
3497 | -def _get_basic_auth(password): |
3498 | - """Build the basic auth for interacting with an offline proxy""" |
3499 | - # XXX twom 2019-03-15 Hardcoded username, awaiting user management |
3500 | - username = 'admin' |
3501 | - credentials = '{}:{}'.format(username, password) |
3502 | - try: |
3503 | - encoded_credentials = base64.b64encode(credentials.encode('UTF-8')) |
3504 | - except UnicodeEncodeError: |
3505 | - logger.error('Unable to encode password to UTF-8') |
3506 | - raise |
3507 | - return 'Basic {}'.format(encoded_credentials.decode()) |
3508 | - |
3509 | - |
3510 | -def _raise_needs_refresh(response): |
3511 | - if (response.status_code == 401 and |
3512 | - response.headers.get('WWW-Authenticate') == ( |
3513 | - 'Macaroon needs_refresh=1')): |
3514 | - raise exceptions.StoreMacaroonNeedsRefresh() |
3515 | - |
3516 | - |
3517 | -def refresh_if_necessary(store, func, *args, **kwargs): |
3518 | - """Make a request, refreshing macaroons if necessary.""" |
3519 | - try: |
3520 | - return func(*args, **kwargs) |
3521 | - except exceptions.StoreMacaroonNeedsRefresh: |
3522 | - unbound_discharge = refresh_sso_discharge( |
3523 | - store.get('sso_url'), store.get('unbound_discharge')) |
3524 | - store.set('unbound_discharge', unbound_discharge) |
3525 | - store.save() |
3526 | - return func(*args, **kwargs) |
3527 | - |
3528 | - |
3529 | -def get_overrides(store, snap_name, series='16', password=None): |
3530 | - """Get all overrides for a snap.""" |
3531 | - overrides_url = urllib.parse.urljoin( |
3532 | - store.get('gw_url'), |
3533 | - '/v2/metadata/overrides/{}'.format(urllib.parse.quote_plus(snap_name))) |
3534 | - headers = { |
3535 | - 'X-Ubuntu-Series': series, |
3536 | - } |
3537 | - if password: |
3538 | - headers['Authorization'] = _get_basic_auth(password) |
3539 | - else: |
3540 | - headers['Authorization'] = _get_macaroon_auth(store) |
3541 | - resp = requests.get(overrides_url, headers=headers) |
3542 | - _raise_needs_refresh(resp) |
3543 | - if resp.status_code != 200: |
3544 | - _print_error_message('get overrides', resp) |
3545 | - resp.raise_for_status() |
3546 | - return resp.json() |
3547 | - |
3548 | - |
3549 | -def set_overrides(store, overrides, password=None): |
3550 | - """Add or remove channel map overrides for a snap.""" |
3551 | - overrides_url = urllib.parse.urljoin( |
3552 | - store.get('gw_url'), '/v2/metadata/overrides') |
3553 | - if password: |
3554 | - headers = {'Authorization': _get_basic_auth(password)} |
3555 | - else: |
3556 | - headers = {'Authorization': _get_macaroon_auth(store)} |
3557 | - resp = requests.post(overrides_url, headers=headers, json=overrides) |
3558 | - _raise_needs_refresh(resp) |
3559 | - if resp.status_code != 200: |
3560 | - _print_error_message('set override', resp) |
3561 | - resp.raise_for_status() |
3562 | - return resp.json() |
3563 | - |
3564 | - |
3565 | -def _print_error_message(action, response): |
3566 | - """Print failure messages from other services in a standard way.""" |
3567 | - logger.error("Failed to %s:", action) |
3568 | - if response.status_code == 500: |
3569 | - logger.error("Server sent 500 response.") |
3570 | - elif response.status_code == 404: |
3571 | - logger.error("Server sent 404 response") |
3572 | - else: |
3573 | - try: |
3574 | - json_document = response.json() |
3575 | - error_list = json_document.get( |
3576 | - 'error-list', json_document.get('error_list', [])) |
3577 | - for error in error_list: |
3578 | - logger.error(error['message']) |
3579 | - except json.JSONDecodeError: |
3580 | - logger.error('=' * 20) |
3581 | - logger.error(response.content.decode('UTF-8', errors='replace')) |
3582 | - logger.error('=' * 20) |
Ok, so this is a big MP to review, which is not great
It might have have been better to split it into 2 or 3, 1 just adding the python-ols snapcraft plugin, one for overrides and one for authentication?
That said, I think the tests are much improved, and the use of the library makes this all much nicer.