Merge ~tonysimpson/snapstore-client:snapstore-library-integration into snapstore-client:master

Proposed by Tony Simpson
Status: Work in progress
Proposed branch: ~tonysimpson/snapstore-client:snapstore-library-integration
Merge into: snapstore-client:master
Diff against target: 3582 lines (+1951/-1049)
19 files modified
Makefile (+32/-12)
dev/null (+0/-185)
requirements.txt (+1/-0)
snap/plugins/_python/__init__.py (+19/-0)
snap/plugins/_python/_pip.py (+575/-0)
snap/plugins/_python/_python_finder.py (+104/-0)
snap/plugins/_python/_sitecustomize.py (+124/-0)
snap/plugins/_python/errors.py (+79/-0)
snap/plugins/python_ols.py (+499/-0)
snap/snapcraft.yaml (+15/-8)
snapstore (+1/-126)
snapstore_client/__main__.py (+136/-0)
snapstore_client/logic/login.py (+36/-59)
snapstore_client/logic/overrides.py (+79/-71)
snapstore_client/logic/push.py (+0/-5)
snapstore_client/logic/tests/test_login.py (+103/-265)
snapstore_client/logic/tests/test_overrides.py (+106/-313)
snapstore_client/logic/tests/utils.py (+42/-0)
snapstore_client/utils.py (+0/-5)
Reviewer Review Type Date Requested Status
Simon Davy (community) Approve
Review via email: mp+371039@code.launchpad.net

Commit message

Integrate snapstore_library auth and overrides

Description of the change

Integrates the snapstore-library authentication and override

Tests have been simplified because snapstore-library has test coverage. Updated tests just cover functionality in CLI code.

snap/plugins/* should be ignore as it is just a copy of code from lp:snapstore-snap

There is a snapstore-library branch in lp:~siab/+git/siab-dependencies that can be used for testing.

CI integration has not been considered yet - a setup-jenkaas would need to be added to the Makefile and would need to pull siab-dependencies

snapstore-library review here https://code.launchpad.net/~tonysimpson/snapstore-library/+git/snapstore-library/+merge/371038

To post a comment you must log in.
Revision history for this message
Simon Davy (bloodearnest) wrote :

Ok, so this is a big MP to review, which is not great

It might have have been better to split it into 2 or 3, 1 just adding the python-ols snapcraft plugin, one for overrides and one for authentication?

That said, I think the tests are much improved, and the use of the library makes this all much nicer.

review: Approve

Unmerged commits

9038b0a... by Tony Simpson

Removed dead code

e980150... by Tony Simpson

Make build process more like snap-store-proxy

Borrow approach from snapstore-snap (snap-store-proxy).
snap/plugins/* is a copy of plugin from snap-store-proxy.

54341a0... by Tony Simpson

Integrate snapstore_library auth and overrides

* Test have been rewritten, still requires a few more I think
* Dead code has not been removed yet
* Has not been user tested

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/Makefile b/Makefile
index d8f1bd4..2266b98 100644
--- a/Makefile
+++ b/Makefile
@@ -1,15 +1,22 @@
1# In a global tmp dir for now so snapcraft doesn't make copies (see1FILE_DEPS = Makefile snap/snapcraft.yaml snapstore snapstore_client/**/*.py dependencies/*.*
2# https://forum.snapcraft.io/t/customizable-ignores/230/12).
3TMPDIR = /tmp/snapstore-client.tmp
4
5SERVICE_PACKAGE = snapstore_client2SERVICE_PACKAGE = snapstore_client
6ENV = $(TMPDIR)/env3ENV = env
7PYTHON3 = $(ENV)/bin/python34PYTHON3 = $(ENV)/bin/python3
8PIP = $(PYTHON3) -m pip5PIP = $(PYTHON3) -m pip
9FLAKE8 = $(ENV)/bin/flake86FLAKE8 = $(ENV)/bin/flake8
107
11DEPENDENCY_REPO ?= lp:~siab/+git/siab-dependencies8DEPENDENCY_REPO ?= lp:~siab/+git/siab-dependencies
12SNAPSTORE_DEPENDENCY_DIR ?= $(TMPDIR)/dependencies9SNAPSTORE_DEPENDENCY_DIR = dependencies
10
11SNAPCRAFT ?= snapcraft
12VIRT := $(shell systemd-detect-virt)
13ifeq ($(VIRT),lxc)
14 SNAPCRAFT_FLAGS ?= --destructive-mode
15else
16 SNAPCRAFT_FLAGS ?= --use-lxd
17endif
18SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT ?= edge
19export PATH := $(PATH):/snap/bin
1320
1421
15$(SNAPSTORE_DEPENDENCY_DIR):22$(SNAPSTORE_DEPENDENCY_DIR):
@@ -26,8 +33,13 @@ $(ENV)/dev: $(ENV)/prod
2633
27bootstrap: $(ENV)/prod34bootstrap: $(ENV)/prod
2835
29snap:36snap-store-proxy-client.snap: $(SNAPSTORE_DEPENDENCY_DIR) $(FILE_DEPS)
30 snapcraft cleanbuild37 SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT=${SNAPCRAFT_BUILD_ENVIRONMENT_CHANNEL_SNAPCRAFT} SNAPCRAFT_ENABLE_ERROR_REPORTING=0 $(SNAPCRAFT) snap $(SNAPCRAFT_FLAGS) --output snap-store-proxy-client.snap
38
39snap: snap-store-proxy-client.snap
40
41install: snap
42 sudo snap install snap-store-proxy-client.snap --dangerous
3143
32test: $(ENV)/dev44test: $(ENV)/dev
33 $(PYTHON3) -m unittest $(TESTS) 2>&145 $(PYTHON3) -m unittest $(TESTS) 2>&1
@@ -49,11 +61,19 @@ coverage: $(ENV)/dev
49 $(PYTHON3) -m coverage html61 $(PYTHON3) -m coverage html
5062
51clean:63clean:
52 rm -rf $(TMPDIR)64ifneq (,$(findstring --destructive-mode, $(SNAPCRAFT_FLAGS)))
65 rm -fr parts
66 rm -fr stage
67else
68 $(SNAPCRAFT) clean $(SNAPCRAFT_FLAGS)
69endif
70 rm -rf env
53 rm -rf dist docs/build71 rm -rf dist docs/build
54 rm -rf .coverage htmlcov72 rm -rf .coverage htmlcov
55 find -name '__pycache__' -print0 | xargs -0 rm -rf73 snapcraft clean
56 find -name '*.~*' -delete74
75fullclean: clean
76 rm -rf dependencies
5777
5878
59.PHONY: bootstrap test lint coverage clean snap docs79.PHONY: bootstrap test lint coverage clean fullclean snap install docs
diff --git a/requirements.txt b/requirements.txt
index 09ae15a..19a551f 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -2,3 +2,4 @@ pymacaroons
2pysha32pysha3
3pyxdg3pyxdg
4requests4requests
5snapstore_library
5\ No newline at end of file6\ No newline at end of file
diff --git a/snap/plugins/_python/__init__.py b/snap/plugins/_python/__init__.py
6new file mode 1006447new file mode 100644
index 0000000..6d6ad33
--- /dev/null
+++ b/snap/plugins/_python/__init__.py
@@ -0,0 +1,19 @@
1# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
2#
3# Copyright (C) 2017 Canonical Ltd
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17from ._pip import Pip # noqa
18from ._python_finder import get_python_command # noqa
19from ._sitecustomize import generate_sitecustomize # noqa
diff --git a/snap/plugins/_python/_pip.py b/snap/plugins/_python/_pip.py
0new file mode 10064420new file mode 100644
index 0000000..9acf3ba
--- /dev/null
+++ b/snap/plugins/_python/_pip.py
@@ -0,0 +1,575 @@
1# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
2#
3# Copyright (C) 2016-2019 Canonical Ltd
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17import collections
18import contextlib
19import json
20import logging
21import os
22import re
23import shutil
24import stat
25import subprocess
26import sys
27import tempfile
28from typing import List, Optional, Sequence, Set
29
30import snapcraft
31from snapcraft import file_utils
32from snapcraft.internal import mangling
33from ._python_finder import get_python_command, get_python_headers, get_python_home
34from . import errors
35
36logger = logging.getLogger(__name__)
37
38
39def _process_common_args(
40 *, constraints: Optional[Set[str]] = None, process_dependency_links: bool = False
41) -> List[str]:
42 args = []
43 if constraints:
44 for constraint in constraints:
45 args.extend(["--constraint", constraint])
46
47 if process_dependency_links:
48 args.append("--process-dependency-links")
49
50 return args
51
52
53def _process_package_args(
54 *, packages: Sequence[str], requirements: Sequence[str], setup_py_dir: str
55) -> List[str]:
56 args = []
57 if requirements:
58 for requirement in requirements:
59 args.extend(["--requirement", requirement])
60
61 if packages:
62 args.extend(packages)
63
64 if setup_py_dir:
65 args.append(".")
66
67 return args
68
69
70def _replicate_owner_mode(path):
71 # Don't bother with a path that doesn't exist or is a symlink. The target
72 # of the symlink will either be updated anyway, or we won't have permission
73 # to change it.
74 if not os.path.exists(path) or os.path.islink(path):
75 return
76
77 file_mode = os.stat(path).st_mode
78
79 # We at least need to write to it to fix shebangs later
80 new_mode = file_mode | stat.S_IWUSR
81
82 # If the owner can execute it, so should everyone else.
83 if file_mode & stat.S_IXUSR:
84 new_mode |= stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH
85
86 # If the owner can read it, so should everyone else
87 if file_mode & stat.S_IRUSR:
88 new_mode |= stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH
89
90 os.chmod(path, new_mode)
91
92
93def _fix_permissions(path):
94 for root, dirs, files in os.walk(path):
95 for filename in files:
96 _replicate_owner_mode(os.path.join(root, filename))
97 for dirname in dirs:
98 _replicate_owner_mode(os.path.join(root, dirname))
99
100
101def _expand_vars(text, project_dir):
102 text = text.replace('$SNAPCRAFT_PROJECT_DIR', project_dir)
103 text = text.replace('${SNAPCRAFT_PROJECT_DIR}', project_dir)
104 return text
105
106
107class Pip:
108 """Wrapper for pip abstracting the args necessary for use in a part.
109
110 This class takes care of fetching pip, setuptools, and wheel, and then
111 simply shells out to pip with the magical arguments necessary to install
112 packages into a part.
113
114 Of particular importance: packages must be downloaded (via download())
115 before they can be installed or have wheels built.
116 """
117
118 def __init__(self, *, python_major_version, part_dir, install_dir, stage_dir, project_dir):
119 """Initialize pip.
120
121 You must call setup() before you can actually use pip.
122
123 :param str python_major_version: The python major version to find (2 or
124 3)
125 :param str part_dir: Path to the part's working area
126 :param str install_dir: Path to the part's install area
127 :param str stage_dir: Path to the staging area
128 :param str project_dir: Path to the project area
129
130 :raises MissingPythonCommandError: If no python could be found in the
131 staging or part's install area.
132 """
133 self._python_major_version = python_major_version
134 self._install_dir = install_dir
135 self._stage_dir = stage_dir
136 self._project_dir = project_dir
137
138 self._python_package_dir = os.path.join(part_dir, "python-packages")
139 os.makedirs(self._python_package_dir, exist_ok=True)
140
141 self.__python_command = None # type:str
142 self.__python_home = None # type: str
143
144 @property
145 def _python_command(self):
146 """Lazily determine the python command required."""
147 if not self.__python_command:
148 self.__python_command = get_python_command(
149 self._python_major_version,
150 stage_dir=self._stage_dir,
151 install_dir=self._install_dir,
152 )
153 return self.__python_command
154
155 @property
156 def _python_home(self):
157 """Lazily determine the correct python home."""
158 if not self.__python_home:
159 self.__python_home = get_python_home(
160 self._python_major_version,
161 stage_dir=self._stage_dir,
162 install_dir=self._install_dir,
163 )
164 return self.__python_home
165
166 def setup(
167 self,
168 no_index: bool = False,
169 find_links: Optional[Sequence[str]] = None
170 ):
171 """Install pip and dependencies.
172
173 Check to see if pip has already been installed. If not, fetch pip,
174 setuptools, and wheel, and install them so they can be used.
175 """
176
177 self._ensure_pip_installed(
178 no_index=no_index,
179 find_links=find_links
180 )
181 self._ensure_wheel_installed(
182 no_index=no_index,
183 find_links=find_links
184 )
185 self._ensure_setuptools_installed(
186 no_index=no_index,
187 find_links=find_links
188 )
189
190 def is_setup(self):
191 """Return true if this class has already been setup."""
192
193 return (
194 self._is_pip_installed()
195 and self._is_wheel_installed()
196 and self._is_setuptools_installed()
197 )
198
199 def _ensure_pip_installed(
200 self,
201 no_index: bool = False,
202 find_links: Optional[Sequence[str]] = None
203 ):
204 # Check to see if we have our own pip. If not, we need to use the pip
205 # on the host (installed via build-packages) to grab our own.
206 if not self._is_pip_installed():
207 logger.info("Fetching and installing pip...")
208
209 real_python_home = self.__python_home
210
211 # Make sure we're using pip from the host. Wrapping this operation
212 # in a try/finally to make sure we revert away from the host's
213 # python at the end.
214 try:
215 self.__python_home = os.path.join(os.path.sep, "usr")
216
217 # Using the host's pip, install our own pip
218 self.download(
219 {"pip"},
220 no_index=no_index,
221 find_links=find_links
222 )
223 self.install({"pip"}, ignore_installed=True)
224 finally:
225 # Now that we have our own pip, reset the python home
226 self.__python_home = real_python_home
227
228 def _ensure_wheel_installed(
229 self,
230 no_index: bool = False,
231 find_links: Optional[Sequence[str]] = None
232 ):
233 if not self._is_wheel_installed():
234 logger.info("Fetching and installing wheel...")
235 self.download(
236 {"wheel"},
237 no_index=no_index,
238 find_links=find_links
239 )
240 self.install({"wheel"}, ignore_installed=True)
241
242 def _ensure_setuptools_installed(
243 self,
244 no_index: bool = False,
245 find_links: Optional[Sequence[str]] = None
246 ):
247 if not self._is_setuptools_installed():
248 logger.info("Fetching and installing setuptools...")
249 self.download(
250 {"setuptools"},
251 no_index=no_index,
252 find_links=find_links
253 )
254 self.install({"setuptools"}, ignore_installed=True)
255
256 def _is_pip_installed(self):
257 try:
258 # We're expecting an error here at least once complaining about
259 # pip not being installed. In order to verify that the error is the
260 # one we think it is, we need to process the stderr. So we'll
261 # redirect it to stdout. If it's not the error we expect, something
262 # is wrong, so re-raise it.
263 #
264 # Using _run_output here so stdout doesn't get printed to the
265 # terminal.
266 self._run_output([], stderr=subprocess.STDOUT)
267 except subprocess.CalledProcessError as e:
268 output = e.output.decode(sys.getfilesystemencoding()).strip()
269 if "no module named pip" in output.lower():
270 return False
271 else:
272 raise e
273 return True
274
275 def _is_wheel_installed(self):
276 return "wheel" in self.list()
277
278 def _is_setuptools_installed(self):
279 return "setuptools" in self.list()
280
281 def download(
282 self,
283 packages,
284 *,
285 setup_py_dir: Optional[str] = None,
286 constraints: Optional[Set[str]] = None,
287 requirements: Optional[Sequence[str]] = None,
288 process_dependency_links: bool = False,
289 no_index: bool = False,
290 find_links: Optional[Sequence[str]] = None
291 ):
292 """Download packages into cache, but do not install them.
293
294 :param iterable packages: Packages to download from index.
295 :param str setup_py_dir: Directory containing setup.py.
296 :param iterable constraints: Collection of paths to constraints files.
297 :param iterable requirements: Collection of paths to requirements
298 files.
299 :param boolean process_dependency_links: Enable the processing of
300 dependency links.
301 :param boolean no_index: Don't hit PyPI
302 :param iterable find_links: Locations of additional repositories
303 """
304 package_args = _process_package_args(
305 packages=packages, requirements=requirements, setup_py_dir=setup_py_dir
306 )
307
308 if not package_args:
309 return # No operation was requested
310
311 args = _process_common_args(
312 process_dependency_links=process_dependency_links, constraints=constraints
313 )
314 if no_index:
315 args.append('--no-index')
316 if find_links:
317 for fl_item in find_links:
318 args.append('--find-links')
319 args.append(_expand_vars(fl_item, self._project_dir))
320
321 # Using pip with a few special parameters:
322 #
323 # --disable-pip-version-check: Don't whine if pip is out-of-date with
324 # the version on pypi.
325 # --dest: Download packages into the directory we've set aside for it.
326 #
327 # For cwd, setup_py_dir will be the actual directory we need to be in
328 # or None.
329 self._run(
330 [
331 "download",
332 "--disable-pip-version-check",
333 "--dest",
334 self._python_package_dir,
335 ]
336 + args
337 + package_args,
338 cwd=setup_py_dir,
339 )
340
341 def install(
342 self,
343 packages,
344 *,
345 setup_py_dir: Optional[str] = None,
346 constraints: Optional[Set[str]] = None,
347 requirements: Optional[Sequence[str]] = None,
348 process_dependency_links: bool = False,
349 upgrade: bool = False,
350 install_deps: bool = True,
351 ignore_installed: bool = False
352 ):
353 """Install packages from cache.
354
355 The packages should have already been downloaded via `download()`.
356
357 :param iterable packages: Packages to install from cache.
358 :param str setup_py_dir: Directory containing setup.py.
359 :param iterable constraints: Collection of paths to constraints files.
360 :param iterable requirements: Collection of paths to requirements
361 files.
362 :param boolean process_dependency_links: Enable the processing of
363 dependency links.
364 :param boolean upgrade: Recursively upgrade packages.
365 :param boolean install_deps: Install package dependencies.
366 :param boolean ignore_installed: Reinstall packages if they're already
367 installed
368 """
369 package_args = _process_package_args(
370 packages=packages, requirements=requirements, setup_py_dir=setup_py_dir
371 )
372
373 if not package_args:
374 return # No operation was requested
375
376 args = _process_common_args(
377 process_dependency_links=process_dependency_links, constraints=constraints
378 )
379
380 if upgrade:
381 args.append("--upgrade")
382
383 if not install_deps:
384 args.append("--no-deps")
385
386 if ignore_installed:
387 args.append("--ignore-installed")
388
389 # Using pip with a few special parameters:
390 #
391 # --user: Install packages to PYTHONUSERBASE, which we've pointed to
392 # the installdir.
393 # --no-index: Don't hit pypi, assume the packages are already
394 # downloaded (i.e. by using `self.download()`)
395 # --find-links: Provide the directory into which the packages should
396 # have already been fetched
397 #
398 # For cwd, setup_py_dir will be the actual directory we need to be in
399 # or None.
400 self._run(
401 [
402 "install",
403 "--user",
404 "--no-index",
405 "--find-links",
406 self._python_package_dir,
407 ]
408 + args
409 + package_args,
410 cwd=setup_py_dir,
411 )
412
413 # Installing with --user results in a directory with 700 permissions.
414 # We need it a bit more open than that, so open it up.
415 _fix_permissions(self._install_dir)
416
417 # Fix all shebangs to use the in-snap python.
418 mangling.rewrite_python_shebangs(self._install_dir)
419
420 def wheel(
421 self,
422 packages,
423 *,
424 setup_py_dir: Optional[str] = None,
425 constraints: Optional[Set[str]] = None,
426 requirements: Optional[Sequence[str]] = None,
427 process_dependency_links: bool = False
428 ):
429 """Build wheels of packages in the cache.
430
431 The packages should have already been downloaded via `download()`.
432
433 :param iterable packages: Packages in cache for which to build wheels.
434 :param str setup_py_dir: Directory containing setup.py.
435 :param iterable constraints: Collection of paths to constraints files.
436 :param iterable requirements: Collection of paths to requirements
437 files.
438 :param boolean process_dependency_links: Enable the processing of
439 dependency links.
440
441 :return: List of paths to each wheel that was built.
442 :rtype: list
443 """
444 package_args = _process_package_args(
445 packages=packages, requirements=requirements, setup_py_dir=setup_py_dir
446 )
447
448 if not package_args:
449 return [] # No operation was requested
450
451 args = _process_common_args(
452 process_dependency_links=process_dependency_links, constraints=constraints
453 )
454
455 wheels = [] # type: List[str]
456 with tempfile.TemporaryDirectory() as temp_dir:
457
458 # Using pip with a few special parameters:
459 #
460 # --no-index: Don't hit pypi, assume the packages are already
461 # downloaded (i.e. by using `self.download()`)
462 # --find-links: Provide the directory into which the packages
463 # should have already been fetched
464 # --wheel-dir: Build wheels into a temporary working area rather
465 # rather than cwd. We'll copy them over. FIXME: We can
466 # probably get away just building them in the package
467 # dir. Try that once this refactor has been validated.
468 self._run(
469 [
470 "wheel",
471 "--no-index",
472 "--find-links",
473 self._python_package_dir,
474 "--wheel-dir",
475 temp_dir,
476 ]
477 + args
478 + package_args,
479 cwd=setup_py_dir,
480 )
481 wheels = os.listdir(temp_dir)
482 for wheel in wheels:
483 file_utils.link_or_copy(
484 os.path.join(temp_dir, wheel),
485 os.path.join(self._python_package_dir, wheel),
486 )
487
488 return [os.path.join(self._python_package_dir, wheel) for wheel in wheels]
489
490 def list(self, *, user=False):
491 """Determine which packages have been installed.
492
493 :param boolean user: Whether or not to limit results to user base.
494
495 :return: Dict of installed python packages and their versions
496 :rtype: dict
497 """
498 command = ["list"]
499 if user:
500 command.append("--user")
501
502 packages = collections.OrderedDict()
503 try:
504 output = self._run_output(command + ["--format=json"])
505 json_output = json.loads(output, object_pairs_hook=collections.OrderedDict)
506 except subprocess.CalledProcessError:
507 # --format requires a newer pip, so fall back to legacy output
508 output = self._run_output(command)
509 json_output = [] # type: List[Dict[str, str]]
510 version_regex = re.compile(r"\((.+)\)")
511 for line in output.splitlines():
512 line = line.split()
513 m = version_regex.search(line[1])
514 if not m:
515 raise errors.PipListInvalidLegacyFormatError(output)
516 json_output.append({"name": line[0], "version": m.group(1)})
517 except json.decoder.JSONDecodeError as e:
518 raise errors.PipListInvalidJsonError(output) from e
519
520 for package in json_output:
521 if "name" not in package:
522 raise errors.PipListMissingFieldError("name", output)
523 if "version" not in package:
524 raise errors.PipListMissingFieldError("version", output)
525 packages[package["name"]] = package["version"]
526 return packages
527
528 def clean_packages(self):
529 """Remove the package cache."""
530 with contextlib.suppress(FileNotFoundError):
531 shutil.rmtree(self._python_package_dir)
532
533 def env(self):
534 """The environment used by pip.
535
536 This function is only useful if you happen to need to call into pip's
537 environment without using the API otherwise made available here (e.g.
538 calling the setup.py directly instead of with pip).
539
540 :return: Dict of the environment necessary to use the pip contained
541 here.
542 :rtype: dict
543 """
544 env = os.environ.copy()
545 env["PYTHONUSERBASE"] = self._install_dir
546 env["PYTHONHOME"] = self._python_home
547
548 env["PATH"] = "{}:{}".format(
549 os.path.join(self._install_dir, "usr", "bin"), os.path.expandvars("$PATH")
550 )
551
552 headers = get_python_headers(
553 self._python_major_version, stage_dir=self._stage_dir
554 )
555 if headers:
556 current_cppflags = env.get("CPPFLAGS", "")
557 env["CPPFLAGS"] = "-I{}".format(headers)
558 if current_cppflags:
559 env["CPPFLAGS"] = "{} {}".format(env["CPPFLAGS"], current_cppflags)
560
561 return env
562
563 def _run(self, args, runner=None, **kwargs):
564 env = self.env()
565
566 # Using None as the default value instead of common.run so we can mock
567 # common.run.
568 if runner is None:
569 runner = snapcraft.internal.common.run
570 return runner(
571 [self._python_command, "-m", "pip"] + list(args), env=env, **kwargs
572 )
573
574 def _run_output(self, args, **kwargs):
575 return self._run(args, runner=snapcraft.internal.common.run_output, **kwargs)
diff --git a/snap/plugins/_python/_python_finder.py b/snap/plugins/_python/_python_finder.py
0new file mode 100644576new file mode 100644
index 0000000..c677cd6
--- /dev/null
+++ b/snap/plugins/_python/_python_finder.py
@@ -0,0 +1,104 @@
1# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
2#
3# Copyright (C) 2017 Canonical Ltd
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17import os
18import glob
19
20from . import errors
21
22
23def get_python_command(python_major_version, *, stage_dir, install_dir):
24 """Find the python command to use, preferring staged over the part.
25
26 We prefer the staged python as opposed to the in-part python in order to
27 support one part that supplies python, with another part built `after` it
28 wanting to use its python.
29
30 :param str python_major_version: The python major version to find (2 or 3)
31 :param str stage_dir: Path to the staging area
32 :param str install_dir: Path to the part's install area
33
34 :return: Path to the python command that was found
35 :rtype: str
36
37 :raises MissingPythonCommandError: If no python could be found in the
38 staging or part's install area.
39 """
40 python_command_name = "python{}".format(python_major_version)
41 python_command = os.path.join("usr", "bin", python_command_name)
42 staged_python = os.path.join(stage_dir, python_command)
43 part_python = os.path.join(install_dir, python_command)
44
45 if os.path.exists(staged_python):
46 return staged_python
47 elif os.path.exists(part_python):
48 return part_python
49 else:
50 raise errors.MissingPythonCommandError(
51 python_command_name, [stage_dir, install_dir]
52 )
53
54
55def get_python_headers(python_major_version, *, stage_dir):
56 """Find the python headers to use, if any, preferring staged over the host.
57
58 We want to make sure we use the headers from the staging area if available,
59 or we may end up building for an older python version than the one we
60 actually want to use.
61
62 :param str python_major_version: The python version to find (2 or 3)
63 :param str stage_dir: Path to the staging area
64
65 :return: Path to the python headers that were found ('' if none)
66 :rtype: str
67 """
68 python_command_name = "python{}".format(python_major_version)
69 base_match = os.path.join("usr", "include", "{}*".format(python_command_name))
70 staged_python = glob.glob(os.path.join(stage_dir, base_match))
71 host_python = glob.glob(os.path.join(os.path.sep, base_match))
72
73 if staged_python:
74 return staged_python[0]
75 elif host_python:
76 return host_python[0]
77 else:
78 return ""
79
80
81def get_python_home(python_major_version, *, stage_dir, install_dir):
82 """Find the correct PYTHONHOME, preferring staged over the part.
83
84 We prefer the staged python as opposed to the in-part python in order to
85 support one part that supplies python, with another part built `after` it
86 wanting to use its python.
87
88 :param str python_major_version: The python major version to find (2 or 3)
89 :param str stage_dir: Path to the staging area
90 :param str install_dir: Path to the part's install area
91
92 :return: Path to the PYTHONHOME that was found
93 :rtype: str
94
95 :raises MissingPythonCommandError: If no python could be found in the
96 staging or part's install area.
97 """
98 python_command = get_python_command(
99 python_major_version, stage_dir=stage_dir, install_dir=install_dir
100 )
101 if python_command.startswith(stage_dir):
102 return os.path.join(stage_dir, "usr")
103 else:
104 return os.path.join(install_dir, "usr")
diff --git a/snap/plugins/_python/_sitecustomize.py b/snap/plugins/_python/_sitecustomize.py
0new file mode 100644105new file mode 100644
index 0000000..aed89ea
--- /dev/null
+++ b/snap/plugins/_python/_sitecustomize.py
@@ -0,0 +1,124 @@
1# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
2#
3# Copyright (C) 2017 Canonical Ltd
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17import contextlib
18import glob
19import os
20from textwrap import dedent
21
22from ._python_finder import get_python_command
23from . import errors
24
25_SITECUSTOMIZE_TEMPLATE = dedent(
26 """\
27 import site
28 import os
29
30 snap_dir = os.getenv("SNAP")
31 snapcraft_stage_dir = os.getenv("SNAPCRAFT_STAGE")
32 snapcraft_part_install = os.getenv("SNAPCRAFT_PART_INSTALL")
33
34 for d in (snap_dir, snapcraft_stage_dir, snapcraft_part_install):
35 if d:
36 site_dir = os.path.join(d, "{site_dir}")
37 site.addsitedir(site_dir)
38
39 if snap_dir:
40 site.ENABLE_USER_SITE = False"""
41)
42
43
44def _get_user_site_dir(python_major_version, *, install_dir):
45 path_glob = os.path.join(
46 install_dir, "lib", "python{}*".format(python_major_version), "site-packages"
47 )
48 user_site_dirs = glob.glob(path_glob)
49 if not user_site_dirs:
50 raise errors.MissingUserSitePackagesError(path_glob)
51
52 return user_site_dirs[0][len(install_dir) + 1 :]
53
54
55def _get_sitecustomize_path(python_major_version, *, stage_dir, install_dir):
56 # Use the part's install_dir unless there's a python in the staging area
57 base_dir = install_dir
58 with contextlib.suppress(errors.MissingPythonCommandError):
59 python_command = get_python_command(
60 python_major_version, stage_dir=stage_dir, install_dir=install_dir
61 )
62 if python_command.startswith(stage_dir):
63 base_dir = stage_dir
64
65 site_py_glob = os.path.join(
66 base_dir, "usr", "lib", "python{}*".format(python_major_version), "site.py"
67 )
68 python_sites = glob.glob(site_py_glob)
69 if not python_sites:
70 raise errors.MissingSitePyError(site_py_glob)
71
72 python_site_dir = os.path.dirname(python_sites[0])
73
74 return os.path.join(
75 install_dir, python_site_dir[len(base_dir) + 1 :], "sitecustomize.py"
76 )
77
78
79def generate_sitecustomize(python_major_version, *, stage_dir, install_dir):
80 """Generate a sitecustomize.py to look in staging, part install, and snap.
81
82 This is done by checking the values of the environment variables $SNAP,
83 $SNAPCRAFT_STAGE, and $SNAPCRAFT_PART_INSTALL. As a result, the same
84 sitecustomize.py works to find packages at both build- and run-time.
85
86 :param str python_major_version: The python major version to use (2 or 3)
87 :param str stage_dir: Path to the staging area
88 :param str install_dir: Path to the part's install area
89
90 :raises MissingUserSitePackagesError: If no user site packages are found in
91 install_dir/lib/pythonX*
92 :raises MissingSitePyError: If no site.py can be found in either the
93 staging area or the part install area.
94 """
95 sitecustomize_path = _get_sitecustomize_path(
96 python_major_version, stage_dir=stage_dir, install_dir=install_dir
97 )
98 os.makedirs(os.path.dirname(sitecustomize_path), exist_ok=True)
99
100 # There may very well already be a sitecustomize.py already there. If so,
101 # get rid of it. Is may be a symlink to another sitecustomize.py, in which
102 # case, we'll get rid of that one as well.
103 if os.path.islink(sitecustomize_path):
104 target_path = os.path.realpath(sitecustomize_path)
105
106 # Only remove the target if it's contained within the install directory
107 if target_path.startswith(os.path.abspath(install_dir) + os.sep):
108 with contextlib.suppress(FileNotFoundError):
109 os.remove(target_path)
110
111 with contextlib.suppress(FileNotFoundError):
112 os.remove(sitecustomize_path)
113
114 # Create our sitecustomize. Python from the archives already has one
115 # which is distro-specific and not needed here, so we truncate it if it's
116 # already there.
117 with open(sitecustomize_path, "w") as f:
118 f.write(
119 _SITECUSTOMIZE_TEMPLATE.format(
120 site_dir=_get_user_site_dir(
121 python_major_version, install_dir=install_dir
122 )
123 )
124 )
diff --git a/snap/plugins/_python/errors.py b/snap/plugins/_python/errors.py
0new file mode 100644125new file mode 100644
index 0000000..87fbcfb
--- /dev/null
+++ b/snap/plugins/_python/errors.py
@@ -0,0 +1,79 @@
1# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
2#
3# Copyright (C) 2017 Canonical Ltd
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17import snapcraft.internal.errors
18import snapcraft.formatting_utils
19
20
21class PythonPluginError(snapcraft.internal.errors.SnapcraftError):
22 pass
23
24
25class MissingPythonCommandError(PythonPluginError):
26
27 fmt = "Unable to find {python_version}, searched: {search_paths}"
28
29 def __init__(self, python_version, search_paths):
30 super().__init__(
31 python_version=python_version,
32 search_paths=snapcraft.formatting_utils.combine_paths(
33 search_paths, "", ":"
34 ),
35 )
36
37
38class MissingUserSitePackagesError(PythonPluginError):
39
40 fmt = "Unable to find user site packages: {site_dir_glob}"
41
42 def __init__(self, site_dir_glob):
43 super().__init__(site_dir_glob=site_dir_glob)
44
45
46class MissingSitePyError(PythonPluginError):
47
48 fmt = "Unable to find site.py: {site_py_glob}"
49
50 def __init__(self, site_py_glob):
51 super().__init__(site_py_glob=site_py_glob)
52
53
54class PipListInvalidLegacyFormatError(PythonPluginError):
55
56 fmt = (
57 "Failed to parse Python package list: "
58 "The returned output is not in the expected format:\n"
59 "{output}"
60 )
61
62 def __init__(self, output):
63 super().__init__(output=output)
64
65
66class PipListInvalidJsonError(PythonPluginError):
67
68 fmt = "Pip packages output isn't valid json: {json!r}"
69
70 def __init__(self, json):
71 super().__init__(json=json)
72
73
74class PipListMissingFieldError(PythonPluginError):
75
76 fmt = "Pip packages json missing {field!r} field: {json!r}"
77
78 def __init__(self, field, json):
79 super().__init__(field=field, json=json)
diff --git a/snap/plugins/python_ols.py b/snap/plugins/python_ols.py
0new file mode 10064480new file mode 100644
index 0000000..95598d1
--- /dev/null
+++ b/snap/plugins/python_ols.py
@@ -0,0 +1,499 @@
1# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
2#
3# Copyright (C) 2016-2018 Canonical Ltd
4#
5# This program is free software: you can redistribute it and/or modify
6# it under the terms of the GNU General Public License version 3 as
7# published by the Free Software Foundation.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16
17"""The python plugin can be used for python 2 or 3 based parts.
18
19It can be used for python projects where you would want to do:
20
21 - import python modules with a requirements.txt
22 - build a python project that has a setup.py
23 - install packages straight from pip
24
25This plugin uses the common plugin keywords as well as those for "sources".
26For more information check the 'plugins' topic for the former and the
27'sources' topic for the latter.
28
29Additionally, this plugin uses the following plugin-specific keywords:
30
31 - requirements:
32 (list of strings)
33 List of paths to requirements files.
34 - constraints:
35 (list of strings)
36 List of paths to constraint files.
37 - process-dependency-links:
38 (bool; default: false)
39 Enable the processing of dependency links in pip, which allow one
40 project to provide places to look for another project
41 - python-packages:
42 (list)
43 A list of dependencies to get from PyPI
44 - python-version:
45 (string; default: python3)
46 The python version to use. Valid options are: python2 and python3
47
48If the plugin finds a python interpreter with a basename that matches
49`python-version` in the <stage> directory on the following fixed path:
50`<stage-dir>/usr/bin/<python-interpreter>` then this interpreter would
51be preferred instead and no interpreter would be brought in through
52`stage-packages` mechanisms.
53"""
54
55import collections
56import contextlib
57import os
58import re
59from shutil import which
60from textwrap import dedent
61from typing import List, Set
62
63import requests
64
65import snapcraft
66from snapcraft.common import isurl
67from snapcraft.internal import errors, mangling
68from snapcraft.internal.errors import SnapcraftPluginCommandError
69import _python
70
71
72class UnsupportedPythonVersionError(snapcraft.internal.errors.SnapcraftError):
73
74 fmt = "Unsupported python version: {python_version!r}"
75
76
77class SnapcraftPluginPythonFileMissing(snapcraft.internal.errors.SnapcraftError):
78
79 fmt = (
80 "Failed to find the referred {plugin_property} file at the given "
81 "path: {plugin_property_value!r}.\n"
82 "Check the property and ensure the file exists."
83 )
84
85 def __init__(self, *, plugin_property, plugin_property_value):
86 super().__init__(
87 plugin_property=plugin_property, plugin_property_value=plugin_property_value
88 )
89
90
91class PythonPlugin(snapcraft.BasePlugin):
92 @classmethod
93 def schema(cls):
94 schema = super().schema()
95 schema["properties"]["requirements"] = {
96 "type": "array",
97 "minitems": 1,
98 "uniqueItems": True,
99 "items": {"type": "string"},
100 "default": [],
101 }
102 schema["properties"]["constraints"] = {
103 "type": "array",
104 "minitems": 1,
105 "uniqueItems": True,
106 "items": {"type": "string"},
107 "default": [],
108 }
109 schema["properties"]["python-packages"] = {
110 "type": "array",
111 "minitems": 1,
112 "uniqueItems": True,
113 "items": {"type": "string"},
114 "default": [],
115 }
116 schema["properties"]["process-dependency-links"] = {
117 "type": "boolean",
118 "default": False,
119 }
120 schema["properties"]["python-version"] = {
121 "type": "string",
122 "default": "python3",
123 "enum": ["python2", "python3"],
124 }
125 schema["properties"]["pip-no-index"] = {
126 "type": "boolean",
127 "default": False,
128 }
129 schema["properties"]["pip-find-links"] = {
130 "type": "array",
131 "items": {"type": "string"},
132 "default": [],
133 }
134 schema["anyOf"] = [{"required": ["source"]}, {"required": ["python-packages"]}]
135
136 return schema
137
138 @classmethod
139 def get_pull_properties(cls):
140 # Inform Snapcraft of the properties associated with pulling. If these
141 # change in the YAML Snapcraft will consider the pull step dirty.
142 return [
143 "requirements",
144 "constraints",
145 "python-packages",
146 "process-dependency-links",
147 "python-version",
148 "pip-no-index",
149 "pip-find-links",
150 ]
151
152 @property
153 def plugin_stage_packages(self):
154 if self.options.python_version == "python2":
155 python_base = "python"
156 elif self.options.python_version == "python3":
157 python_base = "python3"
158
159 if self.project.info.base in ("core", "core16", "core18"):
160 stage_packages = [python_base]
161 else:
162 stage_packages = []
163
164 if self.project.info.base == "core18" and python_base == "python3":
165 stage_packages.append("{}-distutils".format(python_base))
166
167 return stage_packages
168
169 # ignore mypy error: Read-only property cannot override read-write property
170 @property # type: ignore
171 def stage_packages(self):
172 try:
173 _python.get_python_command(
174 self._python_major_version,
175 stage_dir=self.project.stage_dir,
176 install_dir=self.installdir,
177 )
178 except _python.errors.MissingPythonCommandError:
179 return super().stage_packages + self.plugin_stage_packages
180 else:
181 return super().stage_packages
182
183 @property
184 def _pip(self):
185 if not self.__pip:
186 self.__pip = _python.Pip(
187 python_major_version=self._python_major_version,
188 part_dir=self.partdir,
189 install_dir=self.installdir,
190 stage_dir=self.project.stage_dir,
191 project_dir=self.project._project_dir,
192 )
193 return self.__pip
194
195 def __init__(self, name, options, project):
196 super().__init__(name, options, project)
197
198 self._setup_base_tools(project.info.base)
199
200 self._manifest = collections.OrderedDict()
201
202 # Pip requires only the major version of python rather than the command
203 # name like our option requires.
204 match = re.match(r"python(?P<major_version>\d).*", self.options.python_version)
205 if not match:
206 raise UnsupportedPythonVersionError(
207 python_version=self.options.python_version
208 )
209
210 self._python_major_version = match.group("major_version")
211 self.__pip = None
212
213 def _setup_base_tools(self, base):
214 # NOTE: stage-packages are lazily loaded.
215 if base in ("core", "core16", "core18"):
216 if self.options.python_version == "python3":
217 self.build_packages.extend(
218 [
219 "python3-dev",
220 "python3-pip",
221 "python3-pkg-resources",
222 "python3-setuptools",
223 ]
224 )
225 elif self.options.python_version == "python2":
226 self.build_packages.extend(
227 [
228 "python-dev",
229 "python-pip",
230 "python-pkg-resources",
231 "python-setuptools",
232 ]
233 )
234 else:
235 raise errors.PluginBaseError(part_name=self.name, base=base)
236
237 def pull(self):
238 super().pull()
239
240 self._pip.setup(
241 no_index=self.options.pip_no_index,
242 find_links=self.options.pip_find_links
243 )
244
245 with simple_env_bzr(os.path.join(self.installdir, "bin")):
246 # Download this project, using its setup.py if present. This will
247 # also download any python-packages requested.
248 self._download_project()
249
250 def clean_pull(self):
251 super().clean_pull()
252 self._pip.clean_packages()
253
254 def build(self):
255 super().build()
256
257 with simple_env_bzr(os.path.join(self.installdir, "bin")):
258 # Install the packages that have already been downloaded
259 installed_pipy_packages = self._install_project()
260
261 requirements = self._get_list_of_packages_from_property(
262 self.options.requirements
263 )
264 if requirements:
265 self._manifest["requirements-contents"] = requirements
266
267 constraints = self._get_list_of_packages_from_property(self.options.constraints)
268 if constraints:
269 self._manifest["constraints-contents"] = constraints
270
271 self._manifest["python-packages"] = [
272 "{}={}".format(name, installed_pipy_packages[name])
273 for name in installed_pipy_packages
274 ]
275
276 _python.generate_sitecustomize(
277 self._python_major_version,
278 stage_dir=self.project.stage_dir,
279 install_dir=self.installdir,
280 )
281
282 def _find_file(self, *, filename: str) -> str:
283 # source-subdir defaults to ''
284 for basepath in [self.builddir, self.sourcedir]:
285 if basepath == self.sourcedir:
286 # This is overwritten in the base plugin
287 # TODO add consistency
288 source_subdir = self.options.source_subdir
289 else:
290 source_subdir = ""
291 filepath = os.path.join(basepath, source_subdir, filename)
292 if os.path.exists(filepath):
293 return filepath
294
295 return None
296
297 def _get_setup_py_dir(self):
298 setup_py_dir = None
299 setup_py_path = self._find_file(filename="setup.py")
300 if setup_py_path:
301 setup_py_dir = os.path.dirname(setup_py_path)
302
303 return setup_py_dir
304
305 def _get_list_of_packages_from_property(self, property_list: Set[str]) -> List[str]:
306 """Return a sorted list of all packages found in property."""
307 package_list = list() # type: List[str]
308 for entry in property_list:
309 contents = self._get_file_contents(entry)
310 package_list.extend(contents.splitlines())
311 return package_list
312
313 def _get_normalized_property_set(
314 self, property_name, property_list: List[str]
315 ) -> Set[str]:
316 """Return a normalized set from a requirements or constraints list."""
317 normalized = set() # type: Set[str]
318 for entry in property_list:
319 if isurl(entry):
320 normalized.add(entry)
321 else:
322 entry_file = self._find_file(filename=entry)
323 if not entry_file:
324 raise SnapcraftPluginPythonFileMissing(
325 plugin_property=property_name, plugin_property_value=entry
326 )
327 normalized.add(entry_file)
328
329 return normalized
330
331 def _install_wheels(self, wheels):
332 installed = self._pip.list()
333 wheel_names = [os.path.basename(w).split("-")[0] for w in wheels]
334 # we want to avoid installing what is already provided in
335 # stage-packages
336 need_install = [k for k in wheel_names if k not in installed]
337 self._pip.install(
338 need_install,
339 upgrade=True,
340 install_deps=False,
341 process_dependency_links=self.options.process_dependency_links,
342 )
343
344 def _download_project(self):
345 constraints = self._get_normalized_property_set(
346 "constraints", self.options.constraints
347 )
348 requirements = self._get_normalized_property_set(
349 "requirements", self.options.requirements
350 )
351
352
353 self._pip.download(
354 self.options.python_packages,
355 setup_py_dir=None,
356 constraints=constraints,
357 requirements=requirements,
358 process_dependency_links=self.options.process_dependency_links,
359 no_index=self.options.pip_no_index,
360 find_links=self.options.pip_find_links,
361 )
362
363 def _install_project(self):
364 setup_py_dir = self._get_setup_py_dir()
365 constraints = self._get_normalized_property_set(
366 "constraints", self.options.constraints
367 )
368 requirements = self._get_normalized_property_set(
369 "requirements", self.options.requirements
370 )
371
372 # setup.py is handled in a different step as some projects may
373 # need to satisfy dependencies for setup.py to be parsed.
374 wheels = self._pip.wheel(
375 self.options.python_packages,
376 setup_py_dir=None,
377 constraints=constraints,
378 requirements=requirements,
379 process_dependency_links=self.options.process_dependency_links,
380 )
381
382 if wheels:
383 self._install_wheels(wheels)
384
385 if setup_py_dir is not None:
386 self._pip.download(
387 [],
388 setup_py_dir=setup_py_dir,
389 constraints=constraints,
390 requirements=set(),
391 process_dependency_links=self.options.process_dependency_links,
392 no_index=self.options.pip_no_index,
393 find_links=self.options.pip_find_links,
394 )
395 wheels = self._pip.wheel(
396 [],
397 setup_py_dir=setup_py_dir,
398 constraints=constraints,
399 requirements=set(),
400 process_dependency_links=self.options.process_dependency_links,
401 )
402
403 if wheels:
404 self._install_wheels(wheels)
405
406 setup_py_path = os.path.join(setup_py_dir, "setup.py")
407 if os.path.exists(setup_py_path):
408 # pbr and others don't work using `pip install .`
409 # LP: #1670852
410 # There is also a chance that this setup.py is distutils based
411 # in which case we will rely on the `pip install .` ran before
412 # this.
413 with contextlib.suppress(SnapcraftPluginCommandError):
414 self._setup_tools_install(setup_py_path)
415
416 return self._pip.list()
417
418 def _setup_tools_install(self, setup_file):
419 command = [
420 _python.get_python_command(
421 self._python_major_version,
422 stage_dir=self.project.stage_dir,
423 install_dir=self.installdir,
424 ),
425 os.path.basename(setup_file),
426 "--no-user-cfg",
427 "install",
428 "--single-version-externally-managed",
429 "--user",
430 "--record",
431 "install.txt",
432 ]
433 self.run(command, env=self._pip.env(), cwd=os.path.dirname(setup_file))
434
435 # Fix all shebangs to use the in-snap python. The stuff installed from
436 # pip has already been fixed, but anything done in this step has not.
437 mangling.rewrite_python_shebangs(self.installdir)
438
439 def _get_file_contents(self, path):
440 if isurl(path):
441 return requests.get(path).text
442 else:
443 file_path = os.path.join(self.sourcedir, path)
444 with open(file_path) as _file:
445 return _file.read()
446
447 def get_manifest(self):
448 return self._manifest
449
450 def snap_fileset(self):
451 fileset = super().snap_fileset()
452 fileset.append("-bin/pip")
453 fileset.append("-bin/pip2")
454 fileset.append("-bin/pip3")
455 fileset.append("-bin/pip2.7")
456 fileset.append("-bin/pip3.*")
457 fileset.append("-bin/easy_install*")
458 fileset.append("-bin/wheel")
459 # The RECORD files include hashes useful when uninstalling packages.
460 # In the snap they will cause conflicts when more than one part uses
461 # the python plugin.
462 fileset.append("-lib/python*/site-packages/*/RECORD")
463 return fileset
464
465
466@contextlib.contextmanager
467def simple_env_bzr(bin_dir):
468 """Create an appropriate environment to run bzr.
469
470 The python plugin sets up PYTHONUSERBASE and PYTHONHOME which
471 conflicts with bzr when using python3 as those two environment
472 variables will make bzr look for modules in the wrong location.
473 """
474 os.makedirs(bin_dir, exist_ok=True)
475 bzr_bin = os.path.join(bin_dir, "bzr")
476 real_bzr_bin = which("bzr")
477 if real_bzr_bin:
478 exec_line = 'exec {} "$@"'.format(real_bzr_bin)
479 else:
480 exec_line = "echo bzr needs to be in PATH; exit 1"
481 with open(bzr_bin, "w") as f:
482 f.write(
483 dedent(
484 """#!/bin/sh
485 unset PYTHONUSERBASE
486 unset PYTHONHOME
487 {}
488 """.format(
489 exec_line
490 )
491 )
492 )
493 os.chmod(bzr_bin, 0o777)
494 try:
495 yield
496 finally:
497 os.remove(bzr_bin)
498 if not os.listdir(bin_dir):
499 os.rmdir(bin_dir)
diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml
index f234ee1..613affa 100644
--- a/snap/snapcraft.yaml
+++ b/snap/snapcraft.yaml
@@ -1,10 +1,10 @@
1
1name: snap-store-proxy-client2name: snap-store-proxy-client
2version: 1.03version: "1.0"
3summary: Canonical snap store proxy administration client.4summary: Canonical snap store proxy administration client.
4description: |5description: |
5 The Canonical snapstore client is used to manage a snap store proxy.6 The Canonical snapstore client is used to manage a snap store proxy.
67base: core
7# Defaults, but snapcraft prints ugly yellow messages without them.
8confinement: strict8confinement: strict
9grade: stable9grade: stable
1010
@@ -16,17 +16,24 @@ apps:
1616
17parts:17parts:
18 deps:18 deps:
19 plugin: python19 plugin: python-ols
20 requirements: ./requirements.txt20 source: .
21 pip-find-links:
22 - $SNAPCRAFT_PROJECT_DIR/dependencies
23 pip-no-index: true
24 requirements:
25 - ./requirements.txt
21 stage-packages:26 stage-packages:
22 - libsodium1827 - libsodium18
23 source:28 source:
24 plugin: dump29 plugin: dump
25 source: .30 source: .
31 override-build: |
32 #TODO: Would be a lot simpler to make snapstore_client normal
33 # Python package
34 cp -a $SNAPCRAFT_PROJECT_DIR/snapstore $SNAPCRAFT_PART_INSTALL
35 cp -a $SNAPCRAFT_PROJECT_DIR/snapstore_client $SNAPCRAFT_PART_INSTALL
26 stage:36 stage:
27 - ./snapstore37 - ./snapstore
28 - ./snapstore_client/38 - ./snapstore_client/
29 override-prime: |
30 snapcraftctl prime
31 /snap/core/current/usr/bin/python3 -m compileall -q -j0 snapstore_client
3239
diff --git a/snapstore b/snapstore
index cd67b71..4550e58 100755
--- a/snapstore
+++ b/snapstore
@@ -1,132 +1,7 @@
1#!/usr/bin/env python31#!/usr/bin/env python3
2# -*- coding: utf-8 -*-
3# Copyright 2017 Canonical Ltd. This software is licensed under the
4# GNU General Public License version 3 (see the file LICENSE).
5
6"""CLI utility to manage a store-in-a-box installation."""
7
8import argparse
9import logging
10import os
11import sys2import sys
123
13from snapstore_client.cli import configure_logging4from snapstore_client.__main__ import main
14from snapstore_client.logic.login import login
15from snapstore_client.logic.overrides import (
16 delete_override,
17 list_overrides,
18 override,
19)
20from snapstore_client.logic.push import push_snap
21
22
23DEFAULT_SSO_URL = 'https://login.ubuntu.com/'
24
25
26def main():
27 try:
28 configure_logging()
29 args = parse_args()
30 return args.func(args) or 0
31 except KeyboardInterrupt:
32 # use default logger as we can't guarantee that configuration
33 # has completed.
34 logging.error("Operation cancelled")
35 return 1
36
37
38def parse_args():
39 parser = argparse.ArgumentParser()
40 subparsers = parser.add_subparsers(help='sub-command help')
41
42 login_parser = subparsers.add_parser(
43 'login', help='Sign into a store.',
44 formatter_class=argparse.ArgumentDefaultsHelpFormatter)
45 login_parser.add_argument('store_url', help='Store URL')
46 login_parser.add_argument('email', help='Ubuntu One SSO email', nargs='?')
47 login_parser.add_argument('--sso-url', help='Ubuntu One SSO URL',
48 default=DEFAULT_SSO_URL)
49 login_parser.add_argument('--offline', help="Use offline mode interaction",
50 action='store_true')
51 login_parser.set_defaults(func=login)
52
53 list_overrides_parser = subparsers.add_parser(
54 'list-overrides', help='List channel map overrides.',
55 )
56 list_overrides_parser.add_argument(
57 '--series', default='16',
58 help='The series within which to list overrides.')
59 list_overrides_parser.add_argument(
60 'snap_name',
61 help='The name of the snap whose channel map should be listed.')
62 list_overrides_parser.add_argument(
63 '--password',
64 help='Password for interacting with an offline proxy',
65 default=os.environ.get('SNAP_PROXY_PASSWORD')
66 )
67 list_overrides_parser.set_defaults(func=list_overrides)
68
69 override_parser = subparsers.add_parser(
70 'override', help='Set channel map overrides.',
71 )
72 override_parser.add_argument(
73 '--series', default='16',
74 help='The series within which to set overrides.')
75 override_parser.add_argument(
76 'snap_name',
77 help='The name of the snap whose channel map should be modified.')
78 override_parser.add_argument(
79 'channel_map_entries', nargs='+', metavar='channel_map_entry',
80 help='A channel map override, in the form <channel>=<revision>.')
81 override_parser.add_argument(
82 '--password',
83 help='Password for interacting with an offline proxy',
84 default=os.environ.get('SNAP_PROXY_PASSWORD')
85 )
86 override_parser.set_defaults(func=override)
87
88 delete_override_parser = subparsers.add_parser(
89 'delete-override', help='Delete channel map overrides.',
90 )
91 delete_override_parser.add_argument(
92 '--series', default='16',
93 help='The series within which to delete overrides.')
94 delete_override_parser.add_argument(
95 'snap_name',
96 help='The name of the snap whose channel map should be modified.')
97 delete_override_parser.add_argument(
98 'channels', nargs='+', metavar='channel',
99 help='A channel whose overrides should be deleted.')
100 delete_override_parser.add_argument(
101 '--password',
102 help='Password for interacting with an offline proxy',
103 default=os.environ.get('SNAP_PROXY_PASSWORD')
104 )
105 delete_override_parser.set_defaults(func=delete_override)
106
107 push_snap_parser = subparsers.add_parser(
108 'push-snap', help='push a snap to an offline proxy')
109 push_snap_parser.add_argument(
110 'snap_tar_file',
111 help='The .tar.gz file of a bundled downloaded snap')
112 push_snap_parser.add_argument(
113 '--push-channel-map',
114 action='store_true',
115 help="Force push of the channel map,"
116 " removing any existing overrides")
117 push_snap_parser.add_argument(
118 '--password',
119 help='Password for interacting with an offline proxy',
120 default=os.environ.get('SNAP_PROXY_PASSWORD')
121 )
122 push_snap_parser.set_defaults(func=push_snap)
123
124 if len(sys.argv) == 1:
125 # Display help if no arguments are provided.
126 parser.print_help()
127 sys.exit(1)
128
129 return parser.parse_args()
1305
1316
132if __name__ == '__main__':7if __name__ == '__main__':
diff --git a/snapstore_client/__main__.py b/snapstore_client/__main__.py
133new file mode 1006448new file mode 100644
index 0000000..be6029a
--- /dev/null
+++ b/snapstore_client/__main__.py
@@ -0,0 +1,136 @@
1# Copyright 2017 Canonical Ltd. This software is licensed under the
2# GNU General Public License version 3 (see the file LICENSE).
3
4"""CLI utility to manage a store-in-a-box installation."""
5
6import argparse
7import logging
8import os
9import sys
10
11from snapstore_client.cli import configure_logging
12from snapstore_client.logic.login import login
13from snapstore_client.logic.overrides import (
14 delete_override,
15 list_overrides,
16 override,
17)
18from snapstore_client.logic.push import push_snap
19
20
21logger = logging.getLogger(__name__)
22
23
24DEFAULT_SSO_URL = 'https://login.ubuntu.com/'
25
26
27def main():
28 try:
29 configure_logging()
30 args = parse_args()
31 return args.func(args) or 0
32 except Exception as e:
33 logging.error("%s: %s", type(e).__name__, e)
34 except KeyboardInterrupt:
35 # use default logger as we can't guarantee that configuration
36 # has completed.
37 logging.error("Operation cancelled")
38 return 1
39
40
41def parse_args():
42 parser = argparse.ArgumentParser()
43 subparsers = parser.add_subparsers(help='sub-command help')
44
45 login_parser = subparsers.add_parser(
46 'login', help='Sign into a store.',
47 formatter_class=argparse.ArgumentDefaultsHelpFormatter)
48 login_parser.add_argument('store_url', help='Store URL')
49 login_parser.add_argument('email', help='Ubuntu One SSO email', nargs='?')
50 login_parser.add_argument('--sso-url', help='Ubuntu One SSO URL',
51 default=DEFAULT_SSO_URL)
52 login_parser.add_argument('--offline', help="Use offline mode interaction",
53 action='store_true')
54 login_parser.set_defaults(func=login)
55
56 list_overrides_parser = subparsers.add_parser(
57 'list-overrides', help='List channel map overrides.',
58 )
59 list_overrides_parser.add_argument(
60 '--series', default='16',
61 help='The series within which to list overrides.')
62 list_overrides_parser.add_argument(
63 'snap_name',
64 help='The name of the snap whose channel map should be listed.')
65 list_overrides_parser.add_argument(
66 '--password',
67 help='Password for interacting with an offline proxy',
68 default=os.environ.get('SNAP_PROXY_PASSWORD')
69 )
70 list_overrides_parser.set_defaults(func=list_overrides)
71
72 override_parser = subparsers.add_parser(
73 'override', help='Set channel map overrides.',
74 )
75 override_parser.add_argument(
76 '--series', default='16',
77 help='The series within which to set overrides.')
78 override_parser.add_argument(
79 'snap_name',
80 help='The name of the snap whose channel map should be modified.')
81 override_parser.add_argument(
82 'channel_map_entries', nargs='+', metavar='channel_map_entry',
83 help='A channel map override, in the form <channel>=<revision>.')
84 override_parser.add_argument(
85 '--password',
86 help='Password for interacting with an offline proxy',
87 default=os.environ.get('SNAP_PROXY_PASSWORD')
88 )
89 override_parser.set_defaults(func=override)
90
91 delete_override_parser = subparsers.add_parser(
92 'delete-override', help='Delete channel map overrides.',
93 )
94 delete_override_parser.add_argument(
95 '--series', default='16',
96 help='The series within which to delete overrides.')
97 delete_override_parser.add_argument(
98 'snap_name',
99 help='The name of the snap whose channel map should be modified.')
100 delete_override_parser.add_argument(
101 'channels', nargs='+', metavar='channel',
102 help='A channel whose overrides should be deleted.')
103 delete_override_parser.add_argument(
104 '--password',
105 help='Password for interacting with an offline proxy',
106 default=os.environ.get('SNAP_PROXY_PASSWORD')
107 )
108 delete_override_parser.set_defaults(func=delete_override)
109
110 push_snap_parser = subparsers.add_parser(
111 'push-snap', help='push a snap to an offline proxy')
112 push_snap_parser.add_argument(
113 'snap_tar_file',
114 help='The .tar.gz file of a bundled downloaded snap')
115 push_snap_parser.add_argument(
116 '--push-channel-map',
117 action='store_true',
118 help="Force push of the channel map,"
119 " removing any existing overrides")
120 push_snap_parser.add_argument(
121 '--password',
122 help='Password for interacting with an offline proxy',
123 default=os.environ.get('SNAP_PROXY_PASSWORD')
124 )
125 push_snap_parser.set_defaults(func=push_snap)
126
127 if len(sys.argv) == 1:
128 # Display help if no arguments are provided.
129 parser.print_help()
130 sys.exit(1)
131
132 return parser.parse_args()
133
134
135if __name__ == '__main__':
136 sys.exit(main())
diff --git a/snapstore_client/exceptions.py b/snapstore_client/exceptions.py
0deleted file mode 100644137deleted file mode 100644
index 061b5fe..0000000
--- a/snapstore_client/exceptions.py
+++ /dev/null
@@ -1,72 +0,0 @@
1# Copyright 2017 Canonical Ltd. This software is licensed under the
2# GNU General Public License version 3 (see the file LICENSE).
3
4"""Client exceptions."""
5
6
7class ClientError(Exception):
8 """The base class for all user-visible exceptions."""
9
10 @property
11 def fmt(self):
12 raise NotImplementedError
13
14 def __init__(self, **kwargs):
15 for key, value in kwargs.items():
16 setattr(self, key, value)
17
18 def __str__(self):
19 return self.fmt.format(**self.__dict__)
20
21
22class InvalidCredentials(ClientError):
23
24 fmt = 'Invalid credentials: {message}.'
25
26 def __init__(self, message):
27 super().__init__(message=message)
28
29
30class InvalidStoreURL(ClientError):
31
32 fmt = 'Invalid store url: {message}.'
33
34 def __init__(self, message):
35 super().__init__(message=message)
36
37
38class StoreCommunicationError(ClientError):
39
40 fmt = 'Connection error with the store using: {message}.'
41
42 def __init__(self, message):
43 super().__init__(message=message)
44
45
46class StoreMacaroonSSOMismatch(ClientError):
47
48 fmt = 'Root macaroon does not refer to expected SSO host: {sso_host}.'
49
50 def __init__(self, sso_host):
51 super().__init__(sso_host=sso_host)
52
53
54class StoreAuthenticationError(ClientError):
55
56 # No terminating full stop because the message from SSO sometimes
57 # (though not always!) includes one.
58 fmt = 'Authentication error: {message}'
59
60 def __init__(self, message, extra=None):
61 super().__init__(message=message, extra=extra)
62
63
64class StoreTwoFactorAuthenticationRequired(StoreAuthenticationError):
65
66 def __init__(self):
67 super().__init__('Two-factor authentication required.')
68
69
70class StoreMacaroonNeedsRefresh(ClientError):
71
72 fmt = 'Authentication macaroon needs to be refreshed.'
diff --git a/snapstore_client/logic/login.py b/snapstore_client/logic/login.py
index 9550988..1eb92ef 100644
--- a/snapstore_client/logic/login.py
+++ b/snapstore_client/logic/login.py
@@ -2,86 +2,63 @@
22
3import getpass3import getpass
4import logging4import logging
5from urllib.parse import urlparse5from urllib.parse import urljoin
66
7from pymacaroons import Macaroon7import requests
88
9from snapstore_client import (9from snapstore_client import config
10 config,10from snapstore_library import exceptions, authentication
11 exceptions,
12 webservices as ws,
13)
1411
1512
16logger = logging.getLogger(__name__)13logger = logging.getLogger(__name__)
1714
1815
19def _extract_caveat_id(sso_url, root_macaroon):16def set_offline(args):
20 macaroon = Macaroon.deserialize(root_macaroon)17 store_name = "default"
21 sso_host = urlparse(sso_url).netloc18 cfg = config.Config()
22 for caveat in macaroon.caveats:19 store = cfg.store_section(store_name)
23 if caveat.location == sso_host:20 store.set("gw_url", args.store_url)
24 return caveat.caveat_id21 cfg.save()
25 else:22 logger.info("Configured %s as the %s store", args.store_url, store_name)
26 raise exceptions.StoreMacaroonSSOMismatch(sso_host)23 return 0
2724
2825
29def login(args):26def login(args):
30 # TODO: validate these before using to avoid ugly errors.
31 gw_url = args.store_url
32 sso_url = args.sso_url
33
34 if args.offline:27 if args.offline:
35 cfg = config.Config()28 return set_offline(args)
36 store = cfg.store_section('default')
37 store.set('gw_url', gw_url)
38 cfg.save()
39 return
4029
41 logger.info('Enter your Ubuntu One SSO credentials.')30 logger.info("Enter your Ubuntu One SSO credentials.")
42 email = args.email31 email = args.email
43 if not email:32 if not email:
44 email = input('Email: ')33 email = input("Email: ")
45 password = getpass.getpass('Password: ')34 password = getpass.getpass("Password: ")
4635
36 session = requests.Session()
37 auth_url = urljoin(args.store_url, "/v2/auth/issue-store-admin")
38 sso_url = urljoin(args.sso_url, "/api/v2/tokens/discharge")
47 try:39 try:
48 root = ws.issue_store_admin(gw_url)40 root_macaroon, discharge_macaroon = authentication.login(
49 except exceptions.ClientError as e:41 session, auth_url, sso_url, email, password
50 logger.error(str(e))42 )
51 return43 except exceptions.TwoFactorAuthenticationRequired:
52 caveat_id = _extract_caveat_id(sso_url, root)44 one_time_password = input("Second-factor auth: ")
53 try:45 root_macaroon, discharge_macaroon = authentication.login(
54 try:46 session, auth_url, sso_url, email, password, otp=one_time_password
55 unbound_discharge = ws.get_sso_discharge(47 )
56 sso_url, email, password, caveat_id)48 logger.info("Login successful")
57 logger.info('Login successful')
58 except exceptions.StoreTwoFactorAuthenticationRequired:
59 one_time_password = input('Second-factor auth: ')
60 unbound_discharge = ws.get_sso_discharge(
61 sso_url, email, password, caveat_id,
62 one_time_password=one_time_password)
63 logger.info('Login successful')
64 except exceptions.StoreAuthenticationError as e:
65 logger.error('Login failed.')
66 logger.error('%s', e)
67 if e.extra:
68 for key, value in e.extra.items():
69 if isinstance(value, list):
70 value = ' '.join(value)
71 logger.error('%s: %s', key, value)
72 return 1
7349
74 cfg = config.Config()50 cfg = config.Config()
75 # For now, the store is always called "default". In the future we may want51 # For now, the store is always called "default". In the future we may want
76 # to support multiple stores by allowing the user to provide a nice name52 # to support multiple stores by allowing the user to provide a nice name
77 # for a store at login that can be used to select the store for later53 # for a store at login that can be used to select the store for later
78 # operations.54 # operations.
79 store = cfg.store_section('default')55 store_name = "default"
80 store.set('gw_url', gw_url)56 store = cfg.store_section(store_name)
81 store.set('sso_url', sso_url)57 store.set("gw_url", args.store_url)
82 store.set('root', root)58 store.set("sso_url", args.sso_url)
83 store.set('unbound_discharge', unbound_discharge)59 store.set("root", root_macaroon.serialize())
84 store.set('email', email)60 store.set("unbound_discharge", discharge_macaroon.serialize())
61 store.set("email", email)
85 cfg.save()62 cfg.save()
8663 logger.info("Configured %s as the %s store", args.store_url, store_name)
87 return 064 return 0
diff --git a/snapstore_client/logic/overrides.py b/snapstore_client/logic/overrides.py
index d3c8356..aa6d490 100644
--- a/snapstore_client/logic/overrides.py
+++ b/snapstore_client/logic/overrides.py
@@ -1,51 +1,73 @@
1# Copyright 2017 Canonical Ltd.1# Copyright 2017 Canonical Ltd.
22
3import logging3import logging
4from urllib.parse import urljoin
45
5from requests.exceptions import HTTPError6import requests
67
7from snapstore_client import (8from snapstore_client import config
8 config,
9 exceptions,
10 webservices as ws,
11)
12from snapstore_client.presentation_helpers import (9from snapstore_client.presentation_helpers import (
13 channel_map_string_to_tuple,10 channel_map_string_to_tuple,
14 override_to_string,11 override_to_string,
15)12)
16from snapstore_client.utils import (13from snapstore_client.utils import (
17 _check_default_store,14 _check_default_store,
18 _log_authorized_error,
19 _log_credentials_error,
20)15)
16from snapstore_library import authentication, overrides
2117
2218
23logger = logging.getLogger(__name__)19logger = logging.getLogger(__name__)
2420
2521
22# Used by the offline store
23HARDCODED_ADMIN = "admin"
24
25
26def _get_macaroon_auth(session, store):
27 refresh_url = urljoin(store.get("sso_url"), "/api/v2/tokens/refresh")
28 refresh_handler = authentication.MacaroonRefreshHandler(
29 session, refresh_url
30 )
31
32 def discharge_updater(discharge):
33 store.set("unbound_discharge", discharge)
34
35 return authentication.HTTPMacaroonAuth(
36 authentication.deserialize(store.get("root")),
37 authentication.deserialize(store.get("unbound_discharge")),
38 refresh_handler=refresh_handler,
39 dischage_update_handler=discharge_updater,
40 )
41
42
43def _get_auth(session, args, store):
44 if args.password:
45 auth = requests.auth.HTTPBasicAuth(HARDCODED_ADMIN, args.password)
46 else:
47 auth = _get_macaroon_auth(session, store)
48 return auth
49
50
51def _print_overrides(overrides_response):
52 for override in overrides_response["overrides"]:
53 logger.info(override_to_string(override))
54
55
26def list_overrides(args):56def list_overrides(args):
27 cfg = config.Config()57 cfg = config.Config()
28 store = _check_default_store(cfg)58 store = _check_default_store(cfg)
29 if not store:59 if not store:
30 return 160 return 1
3161
32 try:62 session = requests.Session()
33 if args.password:63 auth = _get_auth(session, args, store)
34 response = ws.get_overrides(64 url = urljoin(
35 store, args.snap_name,65 store.get("gw_url"), "/v2/metadata/overrides/{}".format(args.snap_name)
36 series=args.series, password=args.password)66 )
37 else:67 overrides_response = overrides.get_overrides(
38 response = ws.refresh_if_necessary(68 session, url, args.snap_name, auth=auth
39 store, ws.get_overrides,69 )
40 store, args.snap_name, series=args.series)70 _print_overrides(overrides_response)
41 except exceptions.InvalidCredentials as e:
42 _log_credentials_error(e)
43 return 1
44 except HTTPError:
45 _log_authorized_error()
46 return 1
47 for override in response['overrides']:
48 logger.info(override_to_string(override))
4971
5072
51def override(args):73def override(args):
@@ -54,31 +76,24 @@ def override(args):
54 if not store:76 if not store:
55 return 177 return 1
5678
57 overrides = []79 overrides_data = []
58 for channel_map_entry in args.channel_map_entries:80 for channel_map_entry in args.channel_map_entries:
59 channel, revision = channel_map_string_to_tuple(channel_map_entry)81 channel, revision = channel_map_string_to_tuple(channel_map_entry)
60 overrides.append({82 overrides_data.append(
61 'snap_name': args.snap_name,83 {
62 'revision': revision,84 "snap_name": args.snap_name,
63 'channel': channel,85 "revision": revision,
64 'series': args.series,86 "channel": channel,
65 })87 "series": args.series,
66 try:88 }
67 if args.password:89 )
68 response = ws.set_overrides(90 session = requests.Session()
69 store, overrides, password=args.password)91 auth = _get_auth(session, args, store)
70 else:92 url = urljoin(store.get("gw_url"), "/v2/metadata/overrides")
71 response = ws.refresh_if_necessary(93 overrides_response = overrides.set_overrides(
72 store, ws.set_overrides,94 session, url, overrides_data, auth=auth
73 store, overrides)95 )
74 except exceptions.InvalidCredentials as e:96 _print_overrides(overrides_response)
75 _log_credentials_error(e)
76 return 1
77 except HTTPError:
78 _log_authorized_error()
79 return 1
80 for override in response['overrides']:
81 logger.info(override_to_string(override))
8297
8398
84def delete_override(args):99def delete_override(args):
@@ -87,27 +102,20 @@ def delete_override(args):
87 if not store:102 if not store:
88 return 1103 return 1
89104
90 overrides = []105 overrides_data = []
91 for channel in args.channels:106 for channel in args.channels:
92 overrides.append({107 overrides_data.append(
93 'snap_name': args.snap_name,108 {
94 'revision': None,109 "snap_name": args.snap_name,
95 'channel': channel,110 "revision": None,
96 'series': args.series,111 "channel": channel,
97 })112 "series": args.series,
98 try:113 }
99 if args.password:114 )
100 response = ws.set_overrides(115 session = requests.Session()
101 store, overrides, password=args.password)116 auth = _get_auth(session, args, store)
102 else:117 url = urljoin(store.get("gw_url"), "/v2/metadata/overrides")
103 response = ws.refresh_if_necessary(118 overrides_response = overrides.set_overrides(
104 store, ws.set_overrides,119 session, url, overrides_data, auth=auth
105 store, overrides)120 )
106 except exceptions.InvalidCredentials as e:121 _print_overrides(overrides_response)
107 _log_credentials_error(e)
108 return 1
109 except HTTPError:
110 _log_authorized_error()
111 return 1
112 for override in response['overrides']:
113 logger.info(override_to_string(override))
diff --git a/snapstore_client/logic/push.py b/snapstore_client/logic/push.py
index abd06e8..be5fcd4 100644
--- a/snapstore_client/logic/push.py
+++ b/snapstore_client/logic/push.py
@@ -10,12 +10,10 @@ from requests.exceptions import HTTPError
1010
11from snapstore_client import (11from snapstore_client import (
12 config,12 config,
13 exceptions,
14)13)
15from snapstore_client.utils import (14from snapstore_client.utils import (
16 _check_default_store,15 _check_default_store,
17 _log_authorized_error,16 _log_authorized_error,
18 _log_credentials_error,
19)17)
2018
21logger = logging.getLogger(__name__)19logger = logging.getLogger(__name__)
@@ -298,9 +296,6 @@ def push_snap(args):
298 logger.error(296 logger.error(
299 "This command only works with a supplied password"297 "This command only works with a supplied password"
300 " and a proxy in offline mode")298 " and a proxy in offline mode")
301 except exceptions.InvalidCredentials as e:
302 _log_credentials_error(e)
303 return 1
304 except HTTPError:299 except HTTPError:
305 _log_authorized_error()300 _log_authorized_error()
306 return 1301 return 1
diff --git a/snapstore_client/logic/tests/test_login.py b/snapstore_client/logic/tests/test_login.py
index d7854cc..f2fce6e 100644
--- a/snapstore_client/logic/tests/test_login.py
+++ b/snapstore_client/logic/tests/test_login.py
@@ -1,286 +1,124 @@
1# Copyright 2017 Canonical Ltd.1# Copyright 2017 Canonical Ltd.
2from testtools.assertions import assert_that
3from testtools.matchers import Equals
4import pymacaroons
25
3import json6from snapstore_client.logic.tests.utils import RunMainTestCase
4from unittest import mock7from snapstore_library import exceptions
5from urllib.parse import urljoin, urlparse
68
7import fixtures
8from pymacaroons import Macaroon
9import responses
10from testtools import TestCase
11from testtools.matchers import (
12 ContainsDict,
13 Equals,
14 MatchesDict,
15)
169
17from snapstore_client import (10class LoginTests(RunMainTestCase):
18 config,11 def test_offline_login(self):
19 exceptions,12 self.mock_with_side_effect(
20)13 "snapstore_library.authentication.login",
21from snapstore_client.logic.login import login14 AssertionError("should not be called"),
22from snapstore_client.tests import factory
23
24
25class LoginTests(TestCase):
26
27 def setUp(self):
28 super().setUp()
29 self.default_gw_url = 'http://store.local/'
30 self.default_sso_url = 'https://login.staging.ubuntu.com/'
31 self.logger = self.useFixture(fixtures.FakeLogger())
32 self.config_path = self.useFixture(fixtures.TempDir()).path
33 self.useFixture(fixtures.MonkeyPatch(
34 'xdg.BaseDirectory.xdg_config_home', self.config_path))
35 self.useFixture(fixtures.MonkeyPatch(
36 'xdg.BaseDirectory.xdg_config_dirs', [self.config_path]))
37 self.mock_input = self.useFixture(fixtures.MockPatch(
38 'builtins.input')).mock
39 self.mock_getpass = self.useFixture(fixtures.MockPatch(
40 'getpass.getpass')).mock
41
42 def make_responses_callback(self, response_templates):
43 full_responses = []
44 for response in response_templates:
45 status = response.get('status', 200)
46 content_type = 'text/plain'
47 if 'json' in response:
48 content_type = 'application/json'
49 body = json.dumps(response['json'])
50 else:
51 body = response.get('body')
52 full_responses.append(
53 (status, {'Content-Type': content_type}, body))
54 iter_responses = iter(full_responses)
55 return lambda request: next(iter_responses)
56
57 def make_args(self, store_url=None,
58 sso_url=None, email=None, offline=False):
59 return factory.Args(
60 store_url=store_url or self.default_gw_url,
61 sso_url=sso_url or self.default_sso_url,
62 email=email,
63 offline=offline,
64 )15 )
6516
66 def add_issue_store_admin_response(self, *response_templates, gw_url=None):17 exit_status = self.run_main(
67 gw_url = gw_url or self.default_gw_url18 "login", "http://store.local/", "--offline"
68 issue_store_admin_url = urljoin(gw_url, '/v2/auth/issue-store-admin')19 )
69 responses.add_callback(
70 'POST', issue_store_admin_url,
71 self.make_responses_callback(response_templates))
72
73 def add_get_sso_discharge_response(self, *response_templates,
74 sso_url=None):
75 sso_url = sso_url or self.default_sso_url
76 discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge')
77 responses.add_callback(
78 'POST', discharge_url,
79 self.make_responses_callback(response_templates))
80
81 def make_root_macaroon(self, sso_url=None):
82 macaroon = Macaroon()
83 sso_url = sso_url or self.default_sso_url
84 sso_host = urlparse(sso_url).netloc
85 macaroon.add_third_party_caveat(sso_host, 'key', 'payload')
86 return macaroon.serialize()
8720
88 @responses.activate21 assert_that(exit_status, Equals(0))
89 def test_login_sso_mismatch(self):22 assert_that(
90 self.mock_input.return_value = 'user@example.org'23 self.logger.output,
91 self.mock_getpass.return_value = 'secret'24 Equals("Configured http://store.local/ as the default store\n"),
92 macaroon = Macaroon()25 )
93 macaroon.add_third_party_caveat('another.example.com', '', '')
94 self.add_issue_store_admin_response(
95 {'status': 200, 'json': {'macaroon': macaroon.serialize()}})
96 self.add_get_sso_discharge_response({'status': 401})
97 self.assertRaises(
98 exceptions.StoreMacaroonSSOMismatch, login, self.make_args())
9926
100 @responses.activate27 def test_login(self):
101 def test_login_sso_bad_email(self):28 root_macaroon = pymacaroons.Macaroon(location="test1")
102 self.mock_input.return_value = ''29 discharge_macaroon = pymacaroons.Macaroon(location="test2")
103 self.mock_getpass.return_value = ''30 self.mock_with_return(
104 self.add_issue_store_admin_response(31 "snapstore_library.authentication.login",
105 {'status': 200, 'json': {'macaroon': self.make_root_macaroon()}})32 (root_macaroon, discharge_macaroon),
106 auth_error = {33 )
107 'message': 'Invalid request data',34 self.mock_with_return("getpass.getpass", "secret")
108 'extra': {'email': ['Enter a valid email address.']},
109 }
110 self.add_get_sso_discharge_response(
111 {'status': 401, 'json': {'error_list': [auth_error]}})
112 self.assertEqual(1, login(self.make_args()))
113 self.assertEqual(
114 'Enter your Ubuntu One SSO credentials.\n'
115 'Login failed.\n'
116 'Authentication error: Invalid request data\n'
117 'email: Enter a valid email address.\n',
118 self.logger.output)
11935
120 @responses.activate36 exit_status = self.run_main(
121 def test_login_sso_unauthorized(self):37 "login", "http://store.local/", "test@example.com"
122 self.mock_input.return_value = 'user@example.org'38 )
123 self.mock_getpass.return_value = 'secret'
124 self.add_issue_store_admin_response(
125 {'status': 200, 'json': {'macaroon': self.make_root_macaroon()}})
126 auth_error = {'message': 'Provided email/password is not correct.'}
127 self.add_get_sso_discharge_response(
128 {'status': 401, 'json': {'error_list': [auth_error]}})
129 self.assertEqual(1, login(self.make_args()))
130 self.assertEqual(
131 'Enter your Ubuntu One SSO credentials.\n'
132 'Login failed.\n'
133 'Authentication error: Provided email/password is not correct.\n',
134 self.logger.output)
13539
136 @responses.activate40 assert_that(exit_status, Equals(0))
137 def test_login_twofactor_required(self):41 assert_that(
138 self.mock_input.side_effect = ('user@example.org', '123456')42 self.logger.output,
139 self.mock_getpass.return_value = 'secret'43 Equals(
140 root = self.make_root_macaroon()44 "Enter your Ubuntu One SSO credentials.\n"
141 self.add_issue_store_admin_response(45 "Login successful\n"
142 {'status': 200, 'json': {'macaroon': root}})46 "Configured http://store.local/ as the default store\n"
143 self.add_get_sso_discharge_response(47 ),
144 {'status': 401,48 )
145 'json': {'error_list': [{'code': 'twofactor-required'}]}},
146 {'status': 200, 'json': {'discharge_macaroon': 'dummy'}})
147 self.assertEqual(0, login(self.make_args()))
14849
149 self.assertIn(50 def test_login_two_factor_required(self):
150 'Enter your Ubuntu One SSO credentials.', self.logger.output)51 root_macaroon = pymacaroons.Macaroon(location="test1")
151 self.mock_input.assert_has_calls([52 discharge_macaroon = pymacaroons.Macaroon(location="test2")
152 mock.call('Email: '), mock.call('Second-factor auth: ')])
153 self.mock_getpass.assert_called_once_with('Password: ')
154 self.assertEqual(3, len(responses.calls))
155 self.assertEqual({
156 'email': 'user@example.org',
157 'password': 'secret',
158 'caveat_id': 'payload',
159 }, json.loads(responses.calls[1].request.body.decode()))
160 self.assertEqual({
161 'email': 'user@example.org',
162 'password': 'secret',
163 'caveat_id': 'payload',
164 'otp': '123456',
165 }, json.loads(responses.calls[2].request.body.decode()))
166 self.assertThat(config.Config().parser, ContainsDict({
167 'store:default': MatchesDict({
168 'gw_url': Equals(self.default_gw_url),
169 'sso_url': Equals(self.default_sso_url),
170 'root': Equals(root),
171 'unbound_discharge': Equals('dummy'),
172 'email': Equals('user@example.org'),
173 }),
174 }))
17553
176 @responses.activate54 def _login(session, auth_url, sso_url, email, password, otp=None):
177 def test_login_twofactor_not_required(self):55 if otp is None:
178 self.mock_input.return_value = 'user@example.org'56 raise exceptions.TwoFactorAuthenticationRequired()
179 self.mock_getpass.return_value = 'secret'57 assert_that(otp, Equals("123456"))
180 root = self.make_root_macaroon()58 return root_macaroon, discharge_macaroon
181 self.add_issue_store_admin_response(
182 {'status': 200, 'json': {'macaroon': root}})
183 self.add_get_sso_discharge_response(
184 {'status': 200, 'json': {'discharge_macaroon': 'dummy'}})
185 login(self.make_args())
18659
187 self.assertIn(60 self.mock_with_side_effect(
188 'Enter your Ubuntu One SSO credentials.', self.logger.output)61 "snapstore_library.authentication.login", _login
189 self.mock_input.assert_called_once_with('Email: ')62 )
190 self.mock_getpass.assert_called_once_with('Password: ')63 self.mock_with_return("getpass.getpass", "secret")
191 self.assertEqual(2, len(responses.calls))64 self.mock_with_return("builtins.input", "123456")
192 self.assertEqual({
193 'email': 'user@example.org',
194 'password': 'secret',
195 'caveat_id': 'payload',
196 }, json.loads(responses.calls[1].request.body.decode()))
197 self.assertThat(config.Config().parser, ContainsDict({
198 'store:default': MatchesDict({
199 'gw_url': Equals(self.default_gw_url),
200 'sso_url': Equals(self.default_sso_url),
201 'root': Equals(root),
202 'unbound_discharge': Equals('dummy'),
203 'email': Equals('user@example.org'),
204 }),
205 }))
20665
207 @responses.activate66 exit_status = self.run_main(
208 def test_login_with_email(self):67 "login", "http://store.local/", "test@example.com"
209 self.mock_input.side_effect = Exception("shouldn't be called")68 )
210 self.mock_getpass.return_value = 'secret'
211 root = self.make_root_macaroon()
212 self.add_issue_store_admin_response(
213 {'status': 200, 'json': {'macaroon': root}})
214 self.add_get_sso_discharge_response(
215 {'status': 200, 'json': {'discharge_macaroon': 'dummy'}})
216 login(self.make_args(email='user@example.org'))
21769
218 self.assertIn(70 assert_that(exit_status, Equals(0))
219 'Enter your Ubuntu One SSO credentials.', self.logger.output)71 assert_that(
220 self.mock_input.assert_not_called()72 self.logger.output,
221 self.mock_getpass.assert_called_once_with('Password: ')73 Equals(
222 self.assertEqual(2, len(responses.calls))74 "Enter your Ubuntu One SSO credentials.\n"
223 self.assertEqual({75 "Login successful\n"
224 'email': 'user@example.org',76 "Configured http://store.local/ as the default store\n"
225 'password': 'secret',77 ),
226 'caveat_id': 'payload',78 )
227 }, json.loads(responses.calls[1].request.body.decode()))
228 self.assertThat(config.Config().parser, ContainsDict({
229 'store:default': MatchesDict({
230 'gw_url': Equals(self.default_gw_url),
231 'sso_url': Equals(self.default_sso_url),
232 'root': Equals(root),
233 'unbound_discharge': Equals('dummy'),
234 'email': Equals('user@example.org'),
235 }),
236 }))
23779
238 @responses.activate80 def test_login_no_email_arg(self):
239 def test_store_url(self):81 root_macaroon = pymacaroons.Macaroon(location="test1")
240 gw_url = 'http://otherstore.local:1234/'82 discharge_macaroon = pymacaroons.Macaroon(location="test2")
24183
242 self.mock_input.return_value = 'user@example.org'84 def _login(session, auth_url, sso_url, email, password, otp=None):
243 self.mock_getpass.return_value = 'secret'85 assert_that(email, Equals("test@example.com"))
244 root = self.make_root_macaroon()86 return root_macaroon, discharge_macaroon
245 self.add_issue_store_admin_response(
246 {'status': 200, 'json': {'macaroon': root}}, gw_url=gw_url)
247 self.add_get_sso_discharge_response(
248 {'status': 200, 'json': {'discharge_macaroon': 'dummy'}})
249 login(self.make_args(store_url=gw_url))
25087
251 self.assertEqual(2, len(responses.calls))88 self.mock_with_side_effect(
252 self.assertEqual(responses.calls[0].request.url[:len(gw_url)], gw_url)89 "snapstore_library.authentication.login", _login
253 self.assertTrue(90 )
254 responses.calls[1].request.url.startswith(self.default_sso_url))91 self.mock_with_return("builtins.input", "test@example.com")
255 self.assertThat(config.Config().parser, ContainsDict({92 self.mock_with_return("getpass.getpass", "secret")
256 'store:default': ContainsDict({93
257 'gw_url': Equals(gw_url),94 exit_status = self.run_main("login", "http://store.local/")
258 'sso_url': Equals(self.default_sso_url),95
259 }),96 assert_that(exit_status, Equals(0))
260 }))97 assert_that(
98 self.logger.output,
99 Equals(
100 "Enter your Ubuntu One SSO credentials.\n"
101 "Login successful\n"
102 "Configured http://store.local/ as the default store\n"
103 ),
104 )
261105
262 @responses.activate106 def test_login_unhandled_error(self):
263 def test_sso_url(self):107 self.mock_with_side_effect(
264 sso_url = 'https://othersso.local:1234/'108 "snapstore_library.authentication.login",
109 Exception("Some error happened"),
110 )
111 self.mock_with_return("getpass.getpass", "secret")
265112
266 self.mock_input.return_value = 'user@example.org'113 exit_status = self.run_main(
267 self.mock_getpass.return_value = 'secret'114 "login", "http://store.local/", "test@example.com"
268 root = self.make_root_macaroon(sso_url=sso_url)115 )
269 self.add_issue_store_admin_response(
270 {'status': 200, 'json': {'macaroon': root}})
271 self.add_get_sso_discharge_response(
272 {'status': 200, 'json': {'discharge_macaroon': 'dummy'}},
273 sso_url=sso_url)
274 login(self.make_args(sso_url=sso_url))
275116
276 self.assertEqual(2, len(responses.calls))117 assert_that(exit_status, Equals(1))
277 self.assertTrue(118 assert_that(
278 responses.calls[0].request.url.startswith(self.default_gw_url))119 self.logger.output,
279 self.assertEqual(120 Equals(
280 responses.calls[1].request.url[:len(sso_url)], sso_url)121 "Enter your Ubuntu One SSO credentials.\n"
281 self.assertThat(config.Config().parser, ContainsDict({122 "Exception: Some error happened\n"
282 'store:default': ContainsDict({123 ),
283 'gw_url': Equals(self.default_gw_url),124 )
284 'sso_url': Equals(sso_url),
285 }),
286 }))
diff --git a/snapstore_client/logic/tests/test_overrides.py b/snapstore_client/logic/tests/test_overrides.py
index 05e32fc..5d53b48 100644
--- a/snapstore_client/logic/tests/test_overrides.py
+++ b/snapstore_client/logic/tests/test_overrides.py
@@ -1,314 +1,107 @@
1# Copyright 2017 Canonical Ltd.1# Copyright 2017 Canonical Ltd.
22from testtools.assertions import assert_that
3import json3from testtools.matchers import Equals
4from urllib.parse import urljoin4
55from snapstore_client.logic.tests.utils import RunMainTestCase
6import fixtures6
7import responses7
8from testtools import TestCase8class OverridesTests(RunMainTestCase):
99 def test_list_overrides(self):
10from snapstore_client import config10 overrides = {
11from snapstore_client.logic.overrides import (11 "overrides": [
12 delete_override,12 {
13 list_overrides,13 "snap_id": "x" * 32,
14 override,14 "snap_name": "mysnap",
15)15 "revision": 1,
16from snapstore_client.tests import (16 "upstream_revision": 2,
17 factory,17 "channel": "stable",
18 testfixtures,18 "architecture": "amd64",
19)19 "series": 16,
2020 },
2121 {
22class OverridesTests(TestCase):22 "snap_id": "x" * 32,
2323 "snap_name": "mysnap",
24 def test_list_overrides_no_store_config(self):24 "revision": 3,
25 self.useFixture(testfixtures.ConfigFixture(empty=True))25 "upstream_revision": 5,
26 logger = self.useFixture(fixtures.FakeLogger())26 "channel": "foo/stable",
27 rc = list_overrides(factory.Args(snap_name='some-snap', series='16'))27 "architecture": "i386",
28 self.assertEqual(rc, 1)28 "series": 16,
29 self.assertEqual(29 },
30 logger.output,30 ]
31 'No store configuration found. '31 }
32 'Have you run "snap-store-proxy-client login"?\n')32 self.mock_with_return(
3333 "snapstore_library.overrides.get_overrides", overrides
34 @responses.activate34 )
35 def test_list_overrides_online(self):35
36 self.useFixture(testfixtures.ConfigFixture())36 self.run_main_preconfigured("list-overrides", "mysnap")
37 logger = self.useFixture(fixtures.FakeLogger())37
38 snap_id = factory.generate_snap_id()38 assert_that(
39 overrides = [39 self.logger.output,
40 factory.SnapDeviceGateway.Override(40 Equals(
41 snap_id=snap_id, snap_name='mysnap'),41 "mysnap stable amd64 1 (upstream 2)\n"
42 factory.SnapDeviceGateway.Override(42 "mysnap foo/stable i386 3 (upstream 5)\n"
43 snap_id=snap_id, snap_name='mysnap', revision=3,43 ),
44 upstream_revision=4, channel='foo/stable',44 )
45 architecture='i386'),45
46 ]46 def test_overrides(self):
47 # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once47 overrides = {
48 # they exist.48 "overrides": [
49 overrides_url = urljoin(49 {
50 config.Config().store_section('default').get('gw_url'),50 "snap_id": "x" * 32,
51 '/v2/metadata/overrides/mysnap')51 "snap_name": "mysnap",
52 responses.add(52 "revision": 1,
53 'GET', overrides_url, status=200, json={'overrides': overrides})53 "upstream_revision": 6,
5454 "channel": "stable",
55 list_overrides(55 "architecture": "amd64",
56 factory.Args(snap_name='mysnap', series='16', password=False))56 "series": 16,
57 self.assertEqual(57 },
58 'mysnap stable amd64 1 (upstream 2)\n'58 {
59 'mysnap foo/stable i386 3 (upstream 4)\n',59 "snap_id": "x" * 32,
60 logger.output)60 "snap_name": "mysnap",
61 # We shouldn't have Basic Authorization headers, but Macaroon61 "revision": 3,
62 self.assertNotIn(62 "upstream_revision": 7,
63 'Basic',63 "channel": "foo/stable",
64 responses.calls[0].request.headers['Authorization'])64 "architecture": "i386",
6565 "series": 16,
66 @responses.activate66 },
67 def test_list_overrides_offline(self):67 ]
68 self.useFixture(testfixtures.ConfigFixture())68 }
69 logger = self.useFixture(fixtures.FakeLogger())69 self.mock_with_return(
70 snap_id = factory.generate_snap_id()70 "snapstore_library.overrides.set_overrides", overrides
71 overrides = [71 )
72 factory.SnapDeviceGateway.Override(72
73 snap_id=snap_id, snap_name='mysnap'),73 self.run_main_preconfigured(
74 factory.SnapDeviceGateway.Override(74 "override", "mysnap", "foo/stable=3", "stable=1"
75 snap_id=snap_id, snap_name='mysnap', revision=3,75 )
76 upstream_revision=4, channel='foo/stable',76
77 architecture='i386'),77 assert_that(
78 ]78 self.logger.output,
79 # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once79 Equals(
80 # they exist.80 "mysnap stable amd64 1 (upstream 6)\n"
81 overrides_url = urljoin(81 "mysnap foo/stable i386 3 (upstream 7)\n"
82 config.Config().store_section('default').get('gw_url'),82 ),
83 '/v2/metadata/overrides/mysnap')83 )
84 responses.add(84
85 'GET', overrides_url, status=200, json={'overrides': overrides})85 def test_delete_overrides(self):
8686 overrides = {
87 list_overrides(87 "overrides": [
88 factory.Args(snap_name='mysnap', series='16', password='test'))88 {
89 self.assertEqual(89 "snap_id": "x" * 32,
90 'mysnap stable amd64 1 (upstream 2)\n'90 "snap_name": "mysnap",
91 'mysnap foo/stable i386 3 (upstream 4)\n',91 "revision": 1,
92 logger.output)92 "upstream_revision": 6,
93 self.assertEqual(93 "channel": "stable",
94 'Basic YWRtaW46dGVzdA==',94 "architecture": "amd64",
95 responses.calls[0].request.headers['Authorization'])95 "series": 16,
9696 }
97 def test_override_no_store_config(self):97 ]
98 self.useFixture(testfixtures.ConfigFixture(empty=True))98 }
99 logger = self.useFixture(fixtures.FakeLogger())99 self.mock_with_return(
100 rc = override(factory.Args(100 "snapstore_library.overrides.set_overrides", overrides
101 snap_name='some-snap', channel_map_entries=['stable=1'],101 )
102 series='16',102
103 password=False))103 self.run_main_preconfigured("delete-override", "mysnap", "foo/stable")
104 self.assertEqual(rc, 1)104
105 self.assertEqual(105 assert_that(
106 logger.output,106 self.logger.output, Equals("mysnap stable amd64 1 (upstream 6)\n")
107 'No store configuration found. '107 )
108 'Have you run "snap-store-proxy-client login"?\n')
109
110 @responses.activate
111 def test_override_online(self):
112 self.useFixture(testfixtures.ConfigFixture())
113 logger = self.useFixture(fixtures.FakeLogger())
114 snap_id = factory.generate_snap_id()
115 overrides = [
116 factory.SnapDeviceGateway.Override(
117 snap_id=snap_id, snap_name='mysnap'),
118 factory.SnapDeviceGateway.Override(
119 snap_id=snap_id, snap_name='mysnap', revision=3,
120 upstream_revision=4, channel='foo/stable',
121 architecture='i386'),
122 ]
123 # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once
124 # they exist.
125 overrides_url = urljoin(
126 config.Config().store_section('default').get('gw_url'),
127 '/v2/metadata/overrides')
128 responses.add(
129 'POST', overrides_url, status=200, json={'overrides': overrides})
130
131 override(factory.Args(
132 snap_name='mysnap',
133 channel_map_entries=['stable=1', 'foo/stable=3'],
134 series='16',
135 password=False))
136 self.assertEqual([
137 {
138 'snap_name': 'mysnap',
139 'revision': 1,
140 'channel': 'stable',
141 'series': '16',
142 },
143 {
144 'snap_name': 'mysnap',
145 'revision': 3,
146 'channel': 'foo/stable',
147 'series': '16',
148 },
149 ], json.loads(responses.calls[0].request.body.decode()))
150 self.assertEqual(
151 'mysnap stable amd64 1 (upstream 2)\n'
152 'mysnap foo/stable i386 3 (upstream 4)\n',
153 logger.output)
154 # We shouldn't have Basic Authorization headers, but Macaroon
155 self.assertNotIn(
156 'Basic',
157 responses.calls[0].request.headers['Authorization'])
158
159 @responses.activate
160 def test_override_offline(self):
161 self.useFixture(testfixtures.ConfigFixture())
162 logger = self.useFixture(fixtures.FakeLogger())
163 snap_id = factory.generate_snap_id()
164 overrides = [
165 factory.SnapDeviceGateway.Override(
166 snap_id=snap_id, snap_name='mysnap'),
167 factory.SnapDeviceGateway.Override(
168 snap_id=snap_id, snap_name='mysnap', revision=3,
169 upstream_revision=4, channel='foo/stable',
170 architecture='i386'),
171 ]
172 # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once
173 # they exist.
174 overrides_url = urljoin(
175 config.Config().store_section('default').get('gw_url'),
176 '/v2/metadata/overrides')
177 responses.add(
178 'POST', overrides_url, status=200, json={'overrides': overrides})
179
180 override(factory.Args(
181 snap_name='mysnap',
182 channel_map_entries=['stable=1', 'foo/stable=3'],
183 series='16',
184 password='test'))
185 self.assertEqual([
186 {
187 'snap_name': 'mysnap',
188 'revision': 1,
189 'channel': 'stable',
190 'series': '16',
191 },
192 {
193 'snap_name': 'mysnap',
194 'revision': 3,
195 'channel': 'foo/stable',
196 'series': '16',
197 },
198 ], json.loads(responses.calls[0].request.body.decode()))
199 self.assertEqual(
200 'mysnap stable amd64 1 (upstream 2)\n'
201 'mysnap foo/stable i386 3 (upstream 4)\n',
202 logger.output)
203 self.assertEqual(
204 'Basic YWRtaW46dGVzdA==',
205 responses.calls[0].request.headers['Authorization'])
206
207 def test_delete_override_no_store_config(self):
208 self.useFixture(testfixtures.ConfigFixture(empty=True))
209 logger = self.useFixture(fixtures.FakeLogger())
210 rc = delete_override(factory.Args(
211 snap_name='some-snap', channels=['stable'],
212 series='16', password=False))
213 self.assertEqual(rc, 1)
214 self.assertEqual(
215 logger.output,
216 'No store configuration found. '
217 'Have you run "snap-store-proxy-client login"?\n')
218
219 @responses.activate
220 def test_delete_override_online(self):
221 self.useFixture(testfixtures.ConfigFixture())
222 logger = self.useFixture(fixtures.FakeLogger())
223 snap_id = factory.generate_snap_id()
224 overrides = [
225 factory.SnapDeviceGateway.Override(
226 snap_id=snap_id, snap_name='mysnap', revision=None),
227 factory.SnapDeviceGateway.Override(
228 snap_id=snap_id, snap_name='mysnap', revision=None,
229 upstream_revision=4, channel='foo/stable',
230 architecture='i386'),
231 ]
232 # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once
233 # they exist.
234 overrides_url = urljoin(
235 config.Config().store_section('default').get('gw_url'),
236 '/v2/metadata/overrides')
237 responses.add(
238 'POST', overrides_url, status=200, json={'overrides': overrides})
239
240 delete_override(factory.Args(
241 snap_name='mysnap',
242 channels=['stable', 'foo/stable'],
243 series='16',
244 password=False))
245 self.assertEqual([
246 {
247 'snap_name': 'mysnap',
248 'revision': None,
249 'channel': 'stable',
250 'series': '16',
251 },
252 {
253 'snap_name': 'mysnap',
254 'revision': None,
255 'channel': 'foo/stable',
256 'series': '16',
257 },
258 ], json.loads(responses.calls[0].request.body.decode()))
259 self.assertEqual(
260 'mysnap stable amd64 is tracking upstream (revision 2)\n'
261 'mysnap foo/stable i386 is tracking upstream (revision 4)\n',
262 logger.output)
263 # We shouldn't have Basic Authorization headers, but Macaroon
264 self.assertNotIn(
265 'Basic',
266 responses.calls[0].request.headers['Authorization'])
267
268 @responses.activate
269 def test_delete_override_offline(self):
270 self.useFixture(testfixtures.ConfigFixture())
271 logger = self.useFixture(fixtures.FakeLogger())
272 snap_id = factory.generate_snap_id()
273 overrides = [
274 factory.SnapDeviceGateway.Override(
275 snap_id=snap_id, snap_name='mysnap', revision=None),
276 factory.SnapDeviceGateway.Override(
277 snap_id=snap_id, snap_name='mysnap', revision=None,
278 upstream_revision=4, channel='foo/stable',
279 architecture='i386'),
280 ]
281 # XXX cjwatson 2017-06-26: Use acceptable-generated doubles once
282 # they exist.
283 overrides_url = urljoin(
284 config.Config().store_section('default').get('gw_url'),
285 '/v2/metadata/overrides')
286 responses.add(
287 'POST', overrides_url, status=200, json={'overrides': overrides})
288
289 delete_override(factory.Args(
290 snap_name='mysnap',
291 channels=['stable', 'foo/stable'],
292 series='16',
293 password='test'))
294 self.assertEqual([
295 {
296 'snap_name': 'mysnap',
297 'revision': None,
298 'channel': 'stable',
299 'series': '16',
300 },
301 {
302 'snap_name': 'mysnap',
303 'revision': None,
304 'channel': 'foo/stable',
305 'series': '16',
306 },
307 ], json.loads(responses.calls[0].request.body.decode()))
308 self.assertEqual(
309 'mysnap stable amd64 is tracking upstream (revision 2)\n'
310 'mysnap foo/stable i386 is tracking upstream (revision 4)\n',
311 logger.output)
312 self.assertEqual(
313 'Basic YWRtaW46dGVzdA==',
314 responses.calls[0].request.headers['Authorization'])
diff --git a/snapstore_client/logic/tests/utils.py b/snapstore_client/logic/tests/utils.py
315new file mode 100644108new file mode 100644
index 0000000..fd2d596
--- /dev/null
+++ b/snapstore_client/logic/tests/utils.py
@@ -0,0 +1,42 @@
1import sys
2
3import testtools
4import fixtures
5
6from snapstore_client.__main__ import main
7from snapstore_client.tests import testfixtures
8
9
10class RunMainTestCase(testtools.TestCase):
11 def mock(self, function):
12 return self.useFixture(fixtures.MockPatch(function)).mock
13
14 def mock_with_return(self, function, rtn):
15 mock = self.mock(function)
16 mock.return_value = rtn
17 return mock
18
19 def mock_with_side_effect(self, function, side_effect):
20 mock = self.mock(function)
21 mock.side_effect = side_effect
22 return mock
23
24 def _run_main(self, *args, empty_config=True):
25 self.useFixture(testfixtures.ConfigFixture(empty=empty_config))
26 self.mock("snapstore_client.__main__.configure_logging")
27 self.logger = self.useFixture(fixtures.FakeLogger())
28
29 old_argv = sys.argv
30 try:
31 sys.argv = ["snap-store-proxy-client"] + list(args)
32 return main()
33 except SystemExit as se:
34 raise AssertionError("Exited with {}".format(se.code))
35 finally:
36 sys.argv = old_argv
37
38 def run_main(self, *args):
39 return self._run_main(*args, empty_config=True)
40
41 def run_main_preconfigured(self, *args):
42 return self._run_main(*args, empty_config=False)
diff --git a/snapstore_client/tests/test_webservices.py b/snapstore_client/tests/test_webservices.py
0deleted file mode 10064443deleted file mode 100644
index c9c743c..0000000
--- a/snapstore_client/tests/test_webservices.py
+++ /dev/null
@@ -1,228 +0,0 @@
1# Copyright 2017 Canonical Ltd.
2
3import json
4import sys
5from urllib.parse import urljoin
6
7import fixtures
8from requests.exceptions import HTTPError
9import responses
10from testtools import TestCase
11
12from snapstore_client import (
13 config,
14 exceptions,
15 webservices,
16)
17from snapstore_client.tests import (
18 factory,
19 matchers,
20 testfixtures,
21)
22
23if sys.version < '3.6':
24 import sha3 # noqa
25
26
27class WebservicesTests(TestCase):
28
29 def setUp(self):
30 super().setUp()
31 self.config = self.useFixture(testfixtures.ConfigFixture())
32
33 @responses.activate
34 def test_issue_store_admin_success(self):
35 gw_url = 'http://store.local/'
36 issue_store_admin_url = urljoin(gw_url, '/v2/auth/issue-store-admin')
37 responses.add(
38 'POST', issue_store_admin_url, status=200,
39 json={'macaroon': 'dummy'})
40
41 self.assertEqual('dummy', webservices.issue_store_admin(gw_url))
42
43 @responses.activate
44 def test_issue_store_admin_error(self):
45 logger = self.useFixture(fixtures.FakeLogger())
46 gw_url = 'http://store.local/'
47 issue_store_admin_url = urljoin(gw_url, '/v2/auth/issue-store-admin')
48 responses.add(
49 'POST', issue_store_admin_url, status=400,
50 json=factory.APIError.single('Something went wrong').to_dict())
51
52 self.assertRaises(
53 HTTPError, webservices.issue_store_admin, gw_url)
54 self.assertEqual(
55 'Failed to issue store_admin macaroon:\nSomething went wrong\n',
56 logger.output)
57
58 @responses.activate
59 def test_get_sso_discharge_success(self):
60 sso_url = 'http://sso.local/'
61 discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge')
62 responses.add(
63 'POST', discharge_url, status=200,
64 json={'discharge_macaroon': 'dummy'})
65
66 self.assertEqual(
67 'dummy',
68 webservices.get_sso_discharge(
69 sso_url, 'user@example.org', 'secret', 'caveat'))
70 request = responses.calls[0].request
71 self.assertEqual('application/json', request.headers['Content-Type'])
72 self.assertEqual({
73 'email': 'user@example.org',
74 'password': 'secret',
75 'caveat_id': 'caveat',
76 }, json.loads(request.body.decode()))
77
78 @responses.activate
79 def test_get_sso_discharge_success_with_otp(self):
80 sso_url = 'http://sso.local/'
81 discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge')
82 responses.add(
83 'POST', discharge_url, status=200,
84 json={'discharge_macaroon': 'dummy'})
85
86 self.assertEqual(
87 'dummy',
88 webservices.get_sso_discharge(
89 sso_url, 'user@example.org', 'secret', 'caveat',
90 one_time_password='123456'))
91 request = responses.calls[0].request
92 self.assertEqual('application/json', request.headers['Content-Type'])
93 self.assertEqual({
94 'email': 'user@example.org',
95 'password': 'secret',
96 'caveat_id': 'caveat',
97 'otp': '123456',
98 }, json.loads(request.body.decode()))
99
100 @responses.activate
101 def test_get_sso_discharge_twofactor_required(self):
102 sso_url = 'http://sso.local/'
103 discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge')
104 responses.add(
105 'POST', discharge_url, status=401,
106 json={'error_list': [{'code': 'twofactor-required'}]})
107
108 self.assertRaises(
109 exceptions.StoreTwoFactorAuthenticationRequired,
110 webservices.get_sso_discharge,
111 sso_url, 'user@example.org', 'secret', 'caveat')
112
113 @responses.activate
114 def test_get_sso_discharge_structured_error(self):
115 sso_url = 'http://sso.local/'
116 discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge')
117 responses.add(
118 'POST', discharge_url, status=400,
119 json={'error_list': [{'code': 'invalid-request',
120 'message': 'Something went wrong'}]})
121
122 e = self.assertRaises(
123 exceptions.StoreAuthenticationError, webservices.get_sso_discharge,
124 sso_url, 'user@example.org', 'secret', 'caveat')
125 self.assertEqual('Something went wrong', e.message)
126
127 @responses.activate
128 def test_get_sso_discharge_unstructured_error(self):
129 logger = self.useFixture(fixtures.FakeLogger())
130 sso_url = 'http://sso.local/'
131 discharge_url = urljoin(sso_url, '/api/v2/tokens/discharge')
132 responses.add(
133 'POST', discharge_url, status=503, body='Try again later.')
134
135 self.assertRaises(
136 HTTPError, webservices.get_sso_discharge,
137 sso_url, 'user@example.org', 'secret', 'caveat')
138 self.assertEqual(
139 'Failed to get SSO discharge:\n'
140 '====================\n'
141 'Try again later.\n'
142 '====================\n',
143 logger.output)
144
145 @responses.activate
146 def test_get_overrides_success(self):
147 logger = self.useFixture(fixtures.FakeLogger())
148 overrides = [factory.SnapDeviceGateway.Override()]
149 # XXX cjwatson 2017-06-26: Use acceptable-generated double once it
150 # exists.
151 store = config.Config().store_section('default')
152 overrides_url = urljoin(
153 store.get('gw_url'), '/v2/metadata/overrides/mysnap')
154 responses.add('GET', overrides_url, status=200, json=overrides)
155
156 self.assertEqual(overrides, webservices.get_overrides(
157 store, 'mysnap'))
158 request = responses.calls[0].request
159 self.assertThat(
160 request.headers['Authorization'],
161 matchers.MacaroonHeaderVerifies(self.config.key))
162 self.assertNotIn('Failed to get overrides:', logger.output)
163
164 @responses.activate
165 def test_get_overrides_error(self):
166 logger = self.useFixture(fixtures.FakeLogger())
167 store = config.Config().store_section('default')
168 overrides_url = urljoin(
169 store.get('gw_url'), '/v2/metadata/overrides/mysnap')
170 responses.add(
171 'GET', overrides_url, status=400,
172 json=factory.APIError.single('Something went wrong').to_dict())
173
174 self.assertRaises(
175 HTTPError, webservices.get_overrides, store, 'mysnap')
176 self.assertEqual(
177 'Failed to get overrides:\nSomething went wrong\n', logger.output)
178
179 @responses.activate
180 def test_set_overrides_success(self):
181 logger = self.useFixture(fixtures.FakeLogger())
182 override = factory.SnapDeviceGateway.Override()
183 # XXX cjwatson 2017-06-26: Use acceptable-generated double once it
184 # exists.
185 store = config.Config().store_section('default')
186 overrides_url = urljoin(store.get('gw_url'), '/v2/metadata/overrides')
187 responses.add('POST', overrides_url, status=200, json=[override])
188
189 self.assertEqual([override], webservices.set_overrides(
190 store, [{
191 'snap_name': override['snap_name'],
192 'revision': override['revision'],
193 'channel': override['channel'],
194 'series': override['series'],
195 }]))
196 request = responses.calls[0].request
197 self.assertThat(
198 request.headers['Authorization'],
199 matchers.MacaroonHeaderVerifies(self.config.key))
200 self.assertEqual([{
201 'snap_name': override['snap_name'],
202 'revision': override['revision'],
203 'channel': override['channel'],
204 'series': override['series'],
205 }], json.loads(request.body.decode()))
206 self.assertNotIn('Failed to set override:', logger.output)
207
208 @responses.activate
209 def test_set_overrides_error(self):
210 logger = self.useFixture(fixtures.FakeLogger())
211 override = factory.SnapDeviceGateway.Override()
212 # XXX cjwatson 2017-06-26: Use acceptable-generated double once it
213 # exists.
214 store = config.Config().store_section('default')
215 overrides_url = urljoin(store.get('gw_url'), '/v2/metadata/overrides')
216 responses.add(
217 'POST', overrides_url, status=400,
218 json=factory.APIError.single('Something went wrong').to_dict())
219
220 self.assertRaises(HTTPError, lambda: webservices.set_overrides(
221 store, {
222 'snap_name': override['snap_name'],
223 'revision': override['revision'],
224 'channel': override['channel'],
225 'series': override['series'],
226 }))
227 self.assertEqual(
228 'Failed to set override:\nSomething went wrong\n', logger.output)
diff --git a/snapstore_client/utils.py b/snapstore_client/utils.py
index 461a1c3..8b10606 100644
--- a/snapstore_client/utils.py
+++ b/snapstore_client/utils.py
@@ -3,11 +3,6 @@ import logging
3logger = logging.getLogger(__name__)3logger = logging.getLogger(__name__)
44
55
6def _log_credentials_error(e):
7 logger.error('%s', e)
8 logger.error('Try to "snap-store-proxy-client login" again.')
9
10
11def _log_authorized_error():6def _log_authorized_error():
12 logger.error(("Perhaps you have not been registered as an "7 logger.error(("Perhaps you have not been registered as an "
13 "admin with the proxy."))8 "admin with the proxy."))
diff --git a/snapstore_client/webservices.py b/snapstore_client/webservices.py
14deleted file mode 1006449deleted file mode 100644
index 3ebf149..0000000
--- a/snapstore_client/webservices.py
+++ /dev/null
@@ -1,185 +0,0 @@
1# Copyright 2017 Canonical Ltd. This software is licensed under the
2# GNU General Public License version 3 (see the file LICENSE).
3
4import base64
5import json
6import logging
7import urllib.parse
8
9from pymacaroons import Macaroon
10import requests
11
12from snapstore_client import exceptions
13
14
15logger = logging.getLogger(__name__)
16
17
18def issue_store_admin(gw_url):
19 """Ask the store to issue a store_admin macaroon."""
20 issue_store_admin_url = urllib.parse.urljoin(
21 gw_url, '/v2/auth/issue-store-admin')
22 try:
23 resp = requests.post(issue_store_admin_url)
24 except requests.exceptions.ConnectionError:
25 raise exceptions.StoreCommunicationError(gw_url)
26 except requests.exceptions.RequestException:
27 raise exceptions.InvalidStoreURL(gw_url)
28 if resp.status_code == 404:
29 _print_error_message('issue store_admin macaroon', resp)
30 raise exceptions.InvalidStoreURL(gw_url)
31 elif resp.status_code != 200:
32 _print_error_message('issue store_admin macaroon', resp)
33 resp.raise_for_status()
34 return resp.json()['macaroon']
35
36
37def get_sso_discharge(sso_url, email, password, caveat_id,
38 one_time_password=None):
39 discharge_url = urllib.parse.urljoin(
40 sso_url, '/api/v2/tokens/discharge')
41 data = {'email': email, 'password': password, 'caveat_id': caveat_id}
42 if one_time_password is not None:
43 data['otp'] = one_time_password
44 resp = requests.post(
45 discharge_url, headers={'Accept': 'application/json'}, json=data)
46 if not resp.ok:
47 try:
48 error_list = resp.json().get('error_list', [])
49 if resp.status_code == 401:
50 for error in error_list:
51 if error.get('code') == 'twofactor-required':
52 raise exceptions.StoreTwoFactorAuthenticationRequired()
53 if error_list:
54 # Only bother about the first error.
55 error = error_list[0]
56 raise exceptions.StoreAuthenticationError(
57 error['message'], extra=error.get('extra'))
58 except json.JSONDecodeError:
59 pass
60 _print_error_message('get SSO discharge', resp)
61 resp.raise_for_status()
62 return resp.json()['discharge_macaroon']
63
64
65def refresh_sso_discharge(store, unbound_discharge_raw):
66 refresh_url = urllib.parse.urljoin(
67 store.get('sso_url'), '/api/v2/tokens/refresh')
68 data = {'discharge_macaroon': unbound_discharge_raw}
69 resp = requests.post(
70 refresh_url, headers={'Accept': 'application/json'}, json=data)
71 if not resp.ok:
72 _print_error_message('refresh SSO discharge', resp)
73 resp.raise_for_status()
74 return resp.json()['discharge_macaroon']
75
76
77def _deserialize_macaroon(name, value):
78 if value is None:
79 raise exceptions.InvalidCredentials('no {} macaroon'.format(name))
80 try:
81 return Macaroon.deserialize(value)
82 except Exception:
83 raise exceptions.InvalidCredentials(
84 'failed to deserialize {} macaroon'.format(name))
85
86
87def _get_macaroon_auth(store):
88 """Return an Authorization header containing store macaroons."""
89 root_raw = store.get('root')
90 root = _deserialize_macaroon('root', root_raw)
91 unbound_discharge_raw = store.get('unbound_discharge')
92 unbound_discharge = _deserialize_macaroon(
93 'unbound discharge', unbound_discharge_raw)
94 bound_discharge = root.prepare_for_request(unbound_discharge)
95 bound_discharge_raw = bound_discharge.serialize()
96 return 'Macaroon root="{}", discharge="{}"'.format(
97 root_raw, bound_discharge_raw)
98
99
100def _get_basic_auth(password):
101 """Build the basic auth for interacting with an offline proxy"""
102 # XXX twom 2019-03-15 Hardcoded username, awaiting user management
103 username = 'admin'
104 credentials = '{}:{}'.format(username, password)
105 try:
106 encoded_credentials = base64.b64encode(credentials.encode('UTF-8'))
107 except UnicodeEncodeError:
108 logger.error('Unable to encode password to UTF-8')
109 raise
110 return 'Basic {}'.format(encoded_credentials.decode())
111
112
113def _raise_needs_refresh(response):
114 if (response.status_code == 401 and
115 response.headers.get('WWW-Authenticate') == (
116 'Macaroon needs_refresh=1')):
117 raise exceptions.StoreMacaroonNeedsRefresh()
118
119
120def refresh_if_necessary(store, func, *args, **kwargs):
121 """Make a request, refreshing macaroons if necessary."""
122 try:
123 return func(*args, **kwargs)
124 except exceptions.StoreMacaroonNeedsRefresh:
125 unbound_discharge = refresh_sso_discharge(
126 store.get('sso_url'), store.get('unbound_discharge'))
127 store.set('unbound_discharge', unbound_discharge)
128 store.save()
129 return func(*args, **kwargs)
130
131
132def get_overrides(store, snap_name, series='16', password=None):
133 """Get all overrides for a snap."""
134 overrides_url = urllib.parse.urljoin(
135 store.get('gw_url'),
136 '/v2/metadata/overrides/{}'.format(urllib.parse.quote_plus(snap_name)))
137 headers = {
138 'X-Ubuntu-Series': series,
139 }
140 if password:
141 headers['Authorization'] = _get_basic_auth(password)
142 else:
143 headers['Authorization'] = _get_macaroon_auth(store)
144 resp = requests.get(overrides_url, headers=headers)
145 _raise_needs_refresh(resp)
146 if resp.status_code != 200:
147 _print_error_message('get overrides', resp)
148 resp.raise_for_status()
149 return resp.json()
150
151
152def set_overrides(store, overrides, password=None):
153 """Add or remove channel map overrides for a snap."""
154 overrides_url = urllib.parse.urljoin(
155 store.get('gw_url'), '/v2/metadata/overrides')
156 if password:
157 headers = {'Authorization': _get_basic_auth(password)}
158 else:
159 headers = {'Authorization': _get_macaroon_auth(store)}
160 resp = requests.post(overrides_url, headers=headers, json=overrides)
161 _raise_needs_refresh(resp)
162 if resp.status_code != 200:
163 _print_error_message('set override', resp)
164 resp.raise_for_status()
165 return resp.json()
166
167
168def _print_error_message(action, response):
169 """Print failure messages from other services in a standard way."""
170 logger.error("Failed to %s:", action)
171 if response.status_code == 500:
172 logger.error("Server sent 500 response.")
173 elif response.status_code == 404:
174 logger.error("Server sent 404 response")
175 else:
176 try:
177 json_document = response.json()
178 error_list = json_document.get(
179 'error-list', json_document.get('error_list', []))
180 for error in error_list:
181 logger.error(error['message'])
182 except json.JSONDecodeError:
183 logger.error('=' * 20)
184 logger.error(response.content.decode('UTF-8', errors='replace'))
185 logger.error('=' * 20)

Subscribers

People subscribed via source and target branches

to all changes: