Merge lp:~jderose/microfiber/ssl into lp:microfiber

Proposed by Jason Gerard DeRose
Status: Merged
Merged at revision: 142
Proposed branch: lp:~jderose/microfiber/ssl
Merge into: lp:microfiber
Diff against target: 1541 lines (+962/-274)
3 files modified
debian/control (+1/-1)
microfiber.py (+146/-47)
test_microfiber.py (+815/-226)
To merge this branch: bzr merge lp:~jderose/microfiber/ssl
Reviewer Review Type Date Requested Status
James Raymond Approve
Review via email: mp+124643@code.launchpad.net

Description of the change

Changes include:

== Server certs are always verified ==

If you have an HTTPS env['url'], server certs are now always verified.

If you provide no additional config, the default openssl ca_path will be used (this is configured by the openssl packagers and should "just work" between distros that put things in different places).

The above is done with SSLContext.set_default_verify_paths(), BTW. Very handy.

If you provide no additional config, host-name verification is on by default.

== SSL config via env['ssl'] ==

The env['ssl'] extension gives you lots of control over the SSL configuration. If present, it must be a `dict`, for which possible keys include 'ca_file', 'ca_path', 'cert_file', 'key_file', and 'check_hostname'.

For example:

env = {
    'url': 'https://127.0.0.1:6984/',
    'ssl': {
        'ca_file': '/my/ssl/foo.ca',
        'check_hostname': False,
        'cert_file': '/my/ssl/bar.cert',
        'key_file': '/my/ssl/bar.key',
    },
}

If you don't want the default openssl ca_path to be used, you can provide your own 'ca_file' and/or 'ca_path'. When you do this, SSLContext.set_default_verify_paths() isn't called. This is what you want if you're using a private CA like we are (also for unit testing).

To turn off host-name verification, include {'check_hostname': False}. You'll want this off for the sort of P2P stuff we're doing with Avahi. You'll also want it off for unit tests (most likely).

Lastly, to use a client SSL certificate (by which the server can verify the client using whatever CA signed the client cert), provide 'cert_file', and also 'key_file' assuming the private key isn't included in the cert.

== HTTPS/SSL unit tests ==

There are now extensive unit tests for HTTPS/SSL, including tests against live CouchDB instances, thanks to the SSL support now in UserCouch.

Anyone needing such live tests should look at usercouch.misc.TempPKI... it's awesome.

== microfiber.Context ==

I've been meaning to do this for a while. Currently we reuse TCP connections within a single `Server` or `Database` instance, but we don't reuse connections between multiple instances that share the same env.

With this merge, you can do this, but you need to explicitly use the same context, like so:

ctx = Context(env)
foo = Database('foo', ctx=ctx)
bar = Database('bar', ctx=ctx)

So now if you were making serial requests back and forth between foo and bar, the TCP connection is reused, which will greatly improve performance.

But the underlying reason to do this now is this way instances can share the same ssl.SSLContext. This is rather expensive to setup, and would be messy to include in CouchBase.

Note that Server.database() and Database.server() automatically return instances that use the same Context (rather that just the same env).

== IPv6 unit tests, niceties ==

As best as I can tell, Microfiber already perfectly supported IPv6. But now I've added numerous IPv6 tests and made a few things nicer for the brave new IPv6 world.

A remaining issue is that for some reason OAuth isn't working with IPv6 (when the http/https URL contains an IPv6 address). My hunch is CouchDB and Microfiber have different ideas about what the canonical URL is, which is used when computing the OAuth signature.

== Unit test refactor, cleanup ==

I spent some time modernizing some of the unit tests and doing various cleanup.

To post a comment you must log in.
Revision history for this message
James Raymond (jamesmr) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/control'
2--- debian/control 2012-08-08 10:55:22 +0000
3+++ debian/control 2012-09-17 10:39:25 +0000
4@@ -5,7 +5,7 @@
5 Build-Depends: debhelper (>= 9.20120115),
6 python3-all (>= 3.2),
7 python3-sphinx,
8- python3-usercouch (>= 12.08),
9+ python3-usercouch (>= 12.09),
10 Standards-Version: 3.9.3
11 X-Python3-Version: >= 3.2
12 Homepage: https://launchpad.net/microfiber
13
14=== modified file 'microfiber.py'
15--- microfiber.py 2012-09-06 15:24:24 +0000
16+++ microfiber.py 2012-09-17 10:39:25 +0000
17@@ -53,6 +53,7 @@
18 import hmac
19 from urllib.parse import urlparse, urlencode, quote_plus
20 from http.client import HTTPConnection, HTTPSConnection, BadStatusLine
21+import ssl
22 import threading
23 from queue import Queue
24 import math
25@@ -82,7 +83,6 @@
26
27 __version__ = '12.09.0'
28 USER_AGENT = 'microfiber ' + __version__
29-SERVER = 'http://localhost:5984/'
30 DC3_CMD = ('/usr/bin/dc3', 'GetEnv')
31 DMEDIA_CMD = ('/usr/bin/dmedia-cli', 'GetEnv')
32
33@@ -90,6 +90,18 @@
34 RANDOM_BYTES = RANDOM_BITS // 8
35 RANDOM_B32LEN = RANDOM_BITS // 5
36
37+HTTP_IPv4_URL = 'http://127.0.0.1:5984/'
38+HTTPS_IPv4_URL = 'https://127.0.0.1:6984/'
39+HTTP_IPv6_URL = 'http://[::1]:5984/'
40+HTTPS_IPv6_URL = 'https://[::1]:6984/'
41+URL_CONSTANTS = (
42+ HTTP_IPv4_URL,
43+ HTTPS_IPv4_URL,
44+ HTTP_IPv6_URL,
45+ HTTPS_IPv6_URL,
46+)
47+DEFAULT_URL = HTTP_IPv4_URL
48+
49
50 def random_id(numbytes=RANDOM_BYTES):
51 """
52@@ -503,6 +515,121 @@
53 thread.join() # Make sure reader() terminates
54
55
56+def build_ssl_context(config):
57+ ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
58+ ctx.verify_mode = ssl.CERT_REQUIRED
59+
60+ # Configure certificate authorities used to verify server certs
61+ if 'ca_file' in config or 'ca_path' in config:
62+ ctx.load_verify_locations(
63+ cafile=config.get('ca_file'),
64+ capath=config.get('ca_path'),
65+ )
66+ else:
67+ ctx.set_default_verify_paths()
68+
69+ # Configure client certificate, if provided
70+ if 'cert_file' in config:
71+ ctx.load_cert_chain(config['cert_file'],
72+ keyfile=config.get('key_file')
73+ )
74+
75+ return ctx
76+
77+
78+class Context:
79+ """
80+ Reuse TCP connections between multiple `CouchBase` instances.
81+
82+ When making serial requests one after another, you get considerably better
83+ performance when you reuse your ``HTTPConnection`` (or ``HTTPSConnection``).
84+
85+ Individual `Server` and `Database` instances automatically do this: each
86+ thread gets its own thread-local connection that will transparently be
87+ reused.
88+
89+ But often you'll have multiple `Server` and `Database` instances all using
90+ the same *env*, and if you were making requests from one to another (say
91+ copying docs, or saving the same doc to multiple databases), you don't
92+ automatically get connection reuse.
93+
94+ To reuse connections among multiple `CouchBase` instances you need to create
95+ them with the same `Context` instance, like this:
96+
97+ >>> from usercouch.misc import TempCouch
98+ >>> from microfiber import Context, Database
99+ >>> tmpcouch = TempCouch()
100+ >>> env = tmpcouch.bootstrap()
101+ >>> ctx = Context(env)
102+ >>> foo = Database('foo', ctx=ctx)
103+ >>> bar = Database('bar', ctx=ctx)
104+ >>> foo.ctx is bar.ctx
105+ True
106+
107+ However, this database doesn't use the same `Context`, despite having an
108+ identical *env*:
109+
110+ >>> baz = Database('baz', env)
111+ >>> baz.ctx is foo.ctx
112+ False
113+
114+ When connecting to CouchDB via SSL, its highly recommended to use the same
115+ `Context` because that will allow all your SSL connections to reuse the
116+ same ``ssl.SSLContext``.
117+ """
118+ def __init__(self, env=None):
119+ if env is None:
120+ env = DEFAULT_URL
121+ if not isinstance(env, (dict, str)):
122+ raise TypeError(
123+ 'env must be a `dict` or `str`; got {!r}'.format(env)
124+ )
125+ self.env = ({'url': env} if isinstance(env, str) else env)
126+ url = self.env.get('url', DEFAULT_URL)
127+ t = urlparse(url)
128+ if t.scheme not in ('http', 'https'):
129+ raise ValueError(
130+ 'url scheme must be http or https; got {!r}'.format(url)
131+ )
132+ if not t.netloc:
133+ raise ValueError('bad url: {!r}'.format(url))
134+ self.basepath = (t.path if t.path.endswith('/') else t.path + '/')
135+ self.t = t
136+ self.url = self.full_url(self.basepath)
137+ self.threadlocal = threading.local()
138+ if t.scheme == 'https':
139+ ssl_config = self.env.get('ssl', {})
140+ self.ssl_ctx = build_ssl_context(ssl_config)
141+ self.check_hostname = ssl_config.get('check_hostname')
142+
143+ def full_url(self, path):
144+ return ''.join([self.t.scheme, '://', self.t.netloc, path])
145+
146+ def get_connection(self):
147+ if self.t.scheme == 'http':
148+ return HTTPConnection(self.t.netloc)
149+ else:
150+ return HTTPSConnection(self.t.netloc,
151+ context=self.ssl_ctx,
152+ check_hostname=self.check_hostname
153+ )
154+
155+ def get_threadlocal_connection(self):
156+ if not hasattr(self.threadlocal, 'connection'):
157+ self.threadlocal.connection = self.get_connection()
158+ return self.threadlocal.connection
159+
160+ def get_auth_headers(self, method, path, query, testing=None):
161+ if 'oauth' in self.env:
162+ baseurl = self.full_url(path)
163+ return _oauth_header(
164+ self.env['oauth'], method, baseurl, dict(query), testing
165+ )
166+ if 'basic' in self.env:
167+ return _basic_auth_header(self.env['basic'])
168+ return {}
169+
170+
171 class CouchBase(object):
172 """
173 Base class for `Server` and `Database`.
174@@ -533,34 +660,14 @@
175 "CouchBase".
176 """
177
178- def __init__(self, env=SERVER):
179- self.env = ({'url': env} if isinstance(env, str) else env)
180- assert isinstance(self.env, dict)
181- url = self.env.get('url', SERVER)
182- t = urlparse(url)
183- if t.scheme not in ('http', 'https'):
184- raise ValueError(
185- 'url scheme must be http or https: {!r}'.format(url)
186- )
187- if not t.netloc:
188- raise ValueError('bad url: {!r}'.format(url))
189- self.scheme = t.scheme
190- self.netloc = t.netloc
191- self.basepath = (t.path if t.path.endswith('/') else t.path + '/')
192- self.url = self._full_url(self.basepath)
193- self._oauth = self.env.get('oauth')
194- self._basic = self.env.get('basic')
195- self.Conn = (HTTPConnection if t.scheme == 'http' else HTTPSConnection)
196- self._threadlocal = threading.local()
197-
198- @property
199- def conn(self):
200- if not hasattr(self._threadlocal, 'conn'):
201- self._threadlocal.conn = self.Conn(self.netloc)
202- return self._threadlocal.conn
203+ def __init__(self, env=None, ctx=None):
204+ self.ctx = (Context(env) if ctx is None else ctx)
205+ self.env = self.ctx.env
206+ self.basepath = self.ctx.basepath
207+ self.url = self.ctx.url
208
209 def _full_url(self, path):
210- return ''.join([self.scheme, '://', self.netloc, path])
211+ return self.ctx.full_url(path)
212
213 def _request(self, method, parts, options, body=None, headers=None):
214 h = {
215@@ -571,27 +678,22 @@
216 h.update(headers)
217 path = (self.basepath + '/'.join(parts) if parts else self.basepath)
218 query = (tuple(_queryiter(options)) if options else tuple())
219- if self._oauth:
220- baseurl = self._full_url(path)
221- h.update(
222- _oauth_header(self._oauth, method, baseurl, dict(query))
223- )
224- elif self._basic:
225- h.update(_basic_auth_header(self._basic))
226+ h.update(self.ctx.get_auth_headers(method, path, query))
227 if query:
228 path = '?'.join([path, urlencode(query)])
229+ conn = self.ctx.get_threadlocal_connection()
230 for retry in range(2):
231 try:
232- self.conn.request(method, path, body, h)
233- response = self.conn.getresponse()
234+ conn.request(method, path, body, h)
235+ response = conn.getresponse()
236 data = response.read()
237 break
238 except BadStatusLine as e:
239- self.conn.close()
240+ conn.close()
241 if retry == 1:
242 raise e
243 except Exception as e:
244- self.conn.close()
245+ conn.close()
246 raise e
247 if response.status >= 500:
248 raise ServerError(response, data, method, path)
249@@ -759,17 +861,14 @@
250 * Server.database(name) - return a Database instance with server URL
251 """
252
253- def __init__(self, env=SERVER):
254- super().__init__(env)
255-
256 def __repr__(self):
257 return '{}({!r})'.format(self.__class__.__name__, self.url)
258
259 def database(self, name, ensure=False):
260 """
261- Return a new `Database` instance for the database *name*.
262+ Create a `Database` with the same `Context` as this `Server`.
263 """
264- db = Database(name, self.env)
265+ db = Database(name, ctx=self.ctx)
266 if ensure:
267 db.ensure()
268 return db
269@@ -809,8 +908,8 @@
270 * `Database.get_many(doc_ids)` - retrieve many docs at once
271 * `Datebase.view(design, view, **options)` - shortcut method, that's all
272 """
273- def __init__(self, name, env=SERVER):
274- super().__init__(env)
275+ def __init__(self, name, env=None, ctx=None):
276+ super().__init__(env, ctx)
277 self.name = name
278 self.basepath += (name + '/')
279
280@@ -821,9 +920,9 @@
281
282 def server(self):
283 """
284- Return a `Server` instance pointing at the same URL as this database.
285+ Create a `Server` with the same `Context` as this `Database`.
286 """
287- return Server(self.env)
288+ return Server(ctx=self.ctx)
289
290 def ensure(self):
291 """
292
293=== modified file 'test_microfiber.py'
294--- test_microfiber.py 2012-09-06 15:24:24 +0000
295+++ test_microfiber.py 2012-09-17 10:39:25 +0000
296@@ -42,13 +42,10 @@
297 from hashlib import md5
298 from urllib.parse import urlparse, urlencode
299 from http.client import HTTPConnection, HTTPSConnection
300+import ssl
301 import threading
302 from random import SystemRandom
303-
304-try:
305- import usercouch.misc
306-except ImportError:
307- usercouch = None
308+from usercouch.misc import TempCouch, TempPKI
309
310 import microfiber
311 from microfiber import random_id
312@@ -56,13 +53,32 @@
313
314
315 random = SystemRandom()
316-
317-# OAuth test string from http://oauth.net/core/1.0a/#anchor46
318-BASE_STRING = 'GET&http%3A%2F%2Fphotos.example.net%2Fphotos&file%3Dvacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk%26oauth_version%3D1.0%26size%3Doriginal'
319-
320 B32ALPHABET = frozenset('234567ABCDEFGHIJKLMNOPQRSTUVWXYZ')
321
322
323+# OAuth 1.0a test vector from http://oauth.net/core/1.0a/#anchor46
324+
325+SAMPLE_OAUTH_TOKENS = (
326+ ('consumer_secret', 'kd94hf93k423kf44'),
327+ ('token_secret', 'pfkkdhi9sl3r4s00'),
328+ ('consumer_key', 'dpf43f3p2l4k3l03'),
329+ ('token', 'nnch734d00sl2jdk'),
330+)
331+
332+SAMPLE_OAUTH_BASE_STRING = 'GET&http%3A%2F%2Fphotos.example.net%2Fphotos&file%3Dvacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk%26oauth_version%3D1.0%26size%3Doriginal'
333+
334+SAMPLE_OAUTH_AUTHORIZATION = ', '.join([
335+ 'OAuth realm=""',
336+ 'oauth_consumer_key="dpf43f3p2l4k3l03"',
337+ 'oauth_nonce="kllo9940pd9333jh"',
338+ 'oauth_signature="tR3%2BTy81lMeYAr%2FFid0kMTYa%2FWM%3D"',
339+ 'oauth_signature_method="HMAC-SHA1"',
340+ 'oauth_timestamp="1191242096"',
341+ 'oauth_token="nnch734d00sl2jdk"',
342+ 'oauth_version="1.0"',
343+])
344+
345+
346 # A sample view from Dmedia:
347 doc_type = """
348 function(doc) {
349@@ -83,6 +99,14 @@
350 }
351
352
353+def test_id():
354+ """
355+ So we can tell our random test IDs from the ones microfiber.random_id()
356+ makes, we use 160-bit IDs instead of 120-bit.
357+ """
358+ return b32encode(os.urandom(20)).decode('ascii')
359+
360+
361 def is_microfiber_id(_id):
362 assert isinstance(_id, str)
363 return (
364@@ -90,6 +114,9 @@
365 and set(_id).issubset(B32ALPHABET)
366 )
367
368+assert is_microfiber_id(microfiber.random_id())
369+assert not is_microfiber_id(test_id())
370+
371
372 def random_dbname():
373 return 'db-' + microfiber.random_id().lower()
374@@ -109,18 +136,6 @@
375 )
376
377
378-def test_id():
379- """
380- So we can tell our random test IDs from the ones microfiber.random_id()
381- makes, we use 160-bit IDs instead of 120-bit.
382- """
383- return b32encode(os.urandom(20)).decode('ascii')
384-
385-
386-assert is_microfiber_id(microfiber.random_id())
387-assert not is_microfiber_id(test_id())
388-
389-
390 class FakeResponse(object):
391 def __init__(self, status, reason):
392 self.status = status
393@@ -170,7 +185,6 @@
394 microfiber.dumps(doc, pretty=True),
395 '{\n "hello": "мир",\n "welcome": "все"\n}'
396 )
397-
398
399 def test_json_body(self):
400 doc = {
401@@ -300,8 +314,6 @@
402 )
403
404 def test_oauth_base_string(self):
405- f = microfiber._oauth_base_string
406-
407 method = 'GET'
408 url = 'http://photos.example.net/photos'
409 params = {
410@@ -314,48 +326,27 @@
411 'file': 'vacation.jpg',
412 'size': 'original',
413 }
414- self.assertEqual(f(method, url, params), BASE_STRING)
415+ self.assertEqual(
416+ microfiber._oauth_base_string(method, url, params),
417+ SAMPLE_OAUTH_BASE_STRING
418+ )
419
420 def test_oauth_sign(self):
421- f = microfiber._oauth_sign
422-
423- oauth = {
424- 'consumer_secret': 'kd94hf93k423kf44',
425- 'token_secret': 'pfkkdhi9sl3r4s00',
426- }
427+ tokens = dict(SAMPLE_OAUTH_TOKENS)
428 self.assertEqual(
429- f(oauth, BASE_STRING),
430+ microfiber._oauth_sign(tokens, SAMPLE_OAUTH_BASE_STRING),
431 'tR3+Ty81lMeYAr/Fid0kMTYa/WM='
432 )
433
434 def test_oauth_header(self):
435- self.maxDiff = None
436- f = microfiber._oauth_header
437-
438- oauth = {
439- 'consumer_secret': 'kd94hf93k423kf44',
440- 'token_secret': 'pfkkdhi9sl3r4s00',
441- 'consumer_key': 'dpf43f3p2l4k3l03',
442- 'token': 'nnch734d00sl2jdk',
443- }
444+ tokens = dict(SAMPLE_OAUTH_TOKENS)
445 method = 'GET'
446 baseurl = 'http://photos.example.net/photos'
447 query = {'file': 'vacation.jpg', 'size': 'original'}
448 testing = ('1191242096', 'kllo9940pd9333jh')
449-
450- expected = ', '.join([
451- 'OAuth realm=""',
452- 'oauth_consumer_key="dpf43f3p2l4k3l03"',
453- 'oauth_nonce="kllo9940pd9333jh"',
454- 'oauth_signature="tR3%2BTy81lMeYAr%2FFid0kMTYa%2FWM%3D"',
455- 'oauth_signature_method="HMAC-SHA1"',
456- 'oauth_timestamp="1191242096"',
457- 'oauth_token="nnch734d00sl2jdk"',
458- 'oauth_version="1.0"',
459- ])
460 self.assertEqual(
461- f(oauth, method, baseurl, query, testing),
462- {'Authorization': expected},
463+ microfiber._oauth_header(tokens, method, baseurl, query, testing),
464+ {'Authorization': SAMPLE_OAUTH_AUTHORIZATION},
465 )
466
467 def test_basic_auth_header(self):
468@@ -366,6 +357,62 @@
469 {'Authorization': 'Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='}
470 )
471
472+ def test_build_ssl_context(self):
473+ pki = TempPKI(client_pki=True)
474+
475+ # FIXME: We need to add tests for config['ca_path'], but
476+ # `usercouch.sslhelpers` doesn't have the needed helpers yet.
477+
478+ # Empty config, uses openssl default ca_path
479+ ctx = microfiber.build_ssl_context({})
480+ self.assertIsInstance(ctx, ssl.SSLContext)
481+ self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1)
482+ self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED)
483+
484+ # Provide ca_file
485+ config = {
486+ 'ca_file': pki.server_ca.ca_file,
487+ }
488+ ctx = microfiber.build_ssl_context(config)
489+ self.assertIsInstance(ctx, ssl.SSLContext)
490+ self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1)
491+ self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED)
492+
493+ # Provide cert_file and key_file (uses openssl default ca_path)
494+ config = {
495+ 'cert_file': pki.client.cert_file,
496+ 'key_file': pki.client.key_file,
497+ }
498+ ctx = microfiber.build_ssl_context(config)
499+ self.assertIsInstance(ctx, ssl.SSLContext)
500+ self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1)
501+ self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED)
502+
503+ # Provide all three
504+ config = {
505+ 'ca_file': pki.server_ca.ca_file,
506+ 'cert_file': pki.client.cert_file,
507+ 'key_file': pki.client.key_file,
508+ }
509+ ctx = microfiber.build_ssl_context(config)
510+ self.assertIsInstance(ctx, ssl.SSLContext)
511+ self.assertEqual(ctx.protocol, ssl.PROTOCOL_TLSv1)
512+ self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED)
513+
514+ # Provide junk ca_file, make sure ca_file is actually being used
515+ config = {
516+ 'ca_file': pki.server_ca.key_file,
517+ }
518+ with self.assertRaises(ssl.SSLError) as cm:
519+ microfiber.build_ssl_context(config)
520+
521+ # Leave out key_file, make sure cert_file is actually being used
522+ config = {
523+ 'cert_file': pki.client.cert_file,
524+ }
525+ with self.assertRaises(ssl.SSLError) as cm:
526+ microfiber.build_ssl_context(config)
527+
528 def test_replication_body(self):
529 src = test_id()
530 dst = test_id()
531@@ -967,87 +1014,528 @@
532 self.assertIs(inst.rows, rows)
533
534
535-class TestCouchBase(TestCase):
536- klass = microfiber.CouchBase
537-
538+class TestContext(TestCase):
539 def test_init(self):
540+ # Test with bad env type:
541+ bad = [microfiber.DEFAULT_URL]
542+ with self.assertRaises(TypeError) as cm:
543+ microfiber.Context(bad)
544+ self.assertEqual(
545+ str(cm.exception),
546+ 'env must be a `dict` or `str`; got {!r}'.format(bad)
547+ )
548+
549+ # Test with bad URL scheme:
550 bad = 'sftp://localhost:5984/'
551 with self.assertRaises(ValueError) as cm:
552- inst = self.klass(bad)
553+ microfiber.Context(bad)
554 self.assertEqual(
555 str(cm.exception),
556- 'url scheme must be http or https: {!r}'.format(bad)
557+ 'url scheme must be http or https; got {!r}'.format(bad)
558 )
559
560+ # Test with bad URL:
561 bad = 'http:localhost:5984/foo/bar'
562 with self.assertRaises(ValueError) as cm:
563- inst = self.klass(bad)
564+ microfiber.Context(bad)
565 self.assertEqual(
566 str(cm.exception),
567 'bad url: {!r}'.format(bad)
568 )
569
570- inst = self.klass('https://localhost:5984/couch?foo=bar/')
571- self.assertEqual(inst.url, 'https://localhost:5984/couch/')
572- self.assertEqual(inst.basepath, '/couch/')
573- self.assertIsInstance(inst.conn, HTTPSConnection)
574- self.assertIs(inst.Conn, HTTPSConnection)
575- self.assertIsNone(inst._oauth)
576- self.assertIsNone(inst._basic)
577-
578- inst = self.klass('http://localhost:5984?/')
579- self.assertEqual(inst.url, 'http://localhost:5984/')
580- self.assertEqual(inst.basepath, '/')
581- self.assertIsInstance(inst.conn, HTTPConnection)
582- self.assertIs(inst.Conn, HTTPConnection)
583- self.assertIsNone(inst._oauth)
584- self.assertIsNone(inst._basic)
585-
586- inst = self.klass('http://localhost:5001/')
587- self.assertEqual(inst.url, 'http://localhost:5001/')
588- self.assertIsInstance(inst.conn, HTTPConnection)
589- self.assertIs(inst.Conn, HTTPConnection)
590- self.assertIsNone(inst._oauth)
591- self.assertIsNone(inst._basic)
592-
593- inst = self.klass('http://localhost:5002')
594- self.assertEqual(inst.url, 'http://localhost:5002/')
595- self.assertIsInstance(inst.conn, HTTPConnection)
596- self.assertIs(inst.Conn, HTTPConnection)
597- self.assertIsNone(inst._oauth)
598- self.assertIsNone(inst._basic)
599-
600- inst = self.klass('https://localhost:5003/')
601- self.assertEqual(inst.url, 'https://localhost:5003/')
602- self.assertIsInstance(inst.conn, HTTPSConnection)
603- self.assertIs(inst.Conn, HTTPSConnection)
604- self.assertIsNone(inst._oauth)
605- self.assertIsNone(inst._basic)
606-
607- inst = self.klass('https://localhost:5004')
608- self.assertEqual(inst.url, 'https://localhost:5004/')
609- self.assertIsInstance(inst.conn, HTTPSConnection)
610- self.assertIs(inst.Conn, HTTPSConnection)
611- self.assertIsNone(inst._oauth)
612- self.assertIsNone(inst._basic)
613-
614- inst = self.klass({'oauth': 'foo'})
615- self.assertEqual(inst._oauth, 'foo')
616-
617- inst = self.klass({'basic': 'bar'})
618- self.assertEqual(inst._basic, 'bar')
619-
620- def test_conn(self):
621+ # Test with default env:
622+ ctx = microfiber.Context()
623+ self.assertEqual(ctx.env, {'url': microfiber.DEFAULT_URL})
624+ self.assertEqual(ctx.basepath, '/')
625+ self.assertEqual(ctx.t, urlparse(microfiber.DEFAULT_URL))
626+ self.assertEqual(ctx.url, microfiber.DEFAULT_URL)
627+ self.assertIsInstance(ctx.threadlocal, threading.local)
628+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
629+ self.assertFalse(hasattr(ctx, 'check_hostname'))
630+
631+ # Test with an empty env dict:
632+ ctx = microfiber.Context({})
633+ self.assertEqual(ctx.env, {})
634+ self.assertEqual(ctx.basepath, '/')
635+ self.assertEqual(ctx.t, urlparse(microfiber.DEFAULT_URL))
636+ self.assertEqual(ctx.url, microfiber.DEFAULT_URL)
637+ self.assertIsInstance(ctx.threadlocal, threading.local)
638+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
639+ self.assertFalse(hasattr(ctx, 'check_hostname'))
640+
641+ # Test with HTTP IPv4 URLs:
642+ url = 'http://localhost:5984/'
643+ for env in (url, {'url': url}):
644+ ctx = microfiber.Context(env)
645+ self.assertEqual(ctx.env, {'url': 'http://localhost:5984/'})
646+ self.assertEqual(ctx.basepath, '/')
647+ self.assertEqual(ctx.t,
648+ ('http', 'localhost:5984', '/', '', '', '')
649+ )
650+ self.assertEqual(ctx.url, 'http://localhost:5984/')
651+ self.assertIsInstance(ctx.threadlocal, threading.local)
652+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
653+ self.assertFalse(hasattr(ctx, 'check_hostname'))
654+ url = 'http://localhost:5984'
655+ for env in (url, {'url': url}):
656+ ctx = microfiber.Context(env)
657+ self.assertEqual(ctx.env, {'url': 'http://localhost:5984'})
658+ self.assertEqual(ctx.basepath, '/')
659+ self.assertEqual(ctx.t,
660+ ('http', 'localhost:5984', '', '', '', '')
661+ )
662+ self.assertEqual(ctx.url, 'http://localhost:5984/')
663+ self.assertIsInstance(ctx.threadlocal, threading.local)
664+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
665+ self.assertFalse(hasattr(ctx, 'check_hostname'))
666+ url = 'http://localhost:5984/foo/'
667+ for env in (url, {'url': url}):
668+ ctx = microfiber.Context(env)
669+ self.assertEqual(ctx.env, {'url': 'http://localhost:5984/foo/'})
670+ self.assertEqual(ctx.basepath, '/foo/')
671+ self.assertEqual(ctx.t,
672+ ('http', 'localhost:5984', '/foo/', '', '', '')
673+ )
674+ self.assertEqual(ctx.url, 'http://localhost:5984/foo/')
675+ self.assertIsInstance(ctx.threadlocal, threading.local)
676+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
677+ self.assertFalse(hasattr(ctx, 'check_hostname'))
678+ url = 'http://localhost:5984/foo'
679+ for env in (url, {'url': url}):
680+ ctx = microfiber.Context(env)
681+ self.assertEqual(ctx.env, {'url': 'http://localhost:5984/foo'})
682+ self.assertEqual(ctx.basepath, '/foo/')
683+ self.assertEqual(ctx.t,
684+ ('http', 'localhost:5984', '/foo', '', '', '')
685+ )
686+ self.assertEqual(ctx.url, 'http://localhost:5984/foo/')
687+ self.assertIsInstance(ctx.threadlocal, threading.local)
688+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
689+ self.assertFalse(hasattr(ctx, 'check_hostname'))
690+
691+ # Test with HTTP IPv6 URLs:
692+ url = 'http://[::1]:5984/'
693+ for env in (url, {'url': url}):
694+ ctx = microfiber.Context(env)
695+ self.assertEqual(ctx.env, {'url': 'http://[::1]:5984/'})
696+ self.assertEqual(ctx.basepath, '/')
697+ self.assertEqual(ctx.t,
698+ ('http', '[::1]:5984', '/', '', '', '')
699+ )
700+ self.assertEqual(ctx.url, 'http://[::1]:5984/')
701+ self.assertIsInstance(ctx.threadlocal, threading.local)
702+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
703+ self.assertFalse(hasattr(ctx, 'check_hostname'))
704+ url = 'http://[::1]:5984'
705+ for env in (url, {'url': url}):
706+ ctx = microfiber.Context(env)
707+ self.assertEqual(ctx.env, {'url': 'http://[::1]:5984'})
708+ self.assertEqual(ctx.basepath, '/')
709+ self.assertEqual(ctx.t,
710+ ('http', '[::1]:5984', '', '', '', '')
711+ )
712+ self.assertEqual(ctx.url, 'http://[::1]:5984/')
713+ self.assertIsInstance(ctx.threadlocal, threading.local)
714+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
715+ self.assertFalse(hasattr(ctx, 'check_hostname'))
716+ url = 'http://[::1]:5984/foo/'
717+ for env in (url, {'url': url}):
718+ ctx = microfiber.Context(env)
719+ self.assertEqual(ctx.env, {'url': 'http://[::1]:5984/foo/'})
720+ self.assertEqual(ctx.basepath, '/foo/')
721+ self.assertEqual(ctx.t,
722+ ('http', '[::1]:5984', '/foo/', '', '', '')
723+ )
724+ self.assertEqual(ctx.url, 'http://[::1]:5984/foo/')
725+ self.assertIsInstance(ctx.threadlocal, threading.local)
726+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
727+ self.assertFalse(hasattr(ctx, 'check_hostname'))
728+ url = 'http://[::1]:5984/foo'
729+ for env in (url, {'url': url}):
730+ ctx = microfiber.Context(env)
731+ self.assertEqual(ctx.env, {'url': 'http://[::1]:5984/foo'})
732+ self.assertEqual(ctx.basepath, '/foo/')
733+ self.assertEqual(ctx.t,
734+ ('http', '[::1]:5984', '/foo', '', '', '')
735+ )
736+ self.assertEqual(ctx.url, 'http://[::1]:5984/foo/')
737+ self.assertIsInstance(ctx.threadlocal, threading.local)
738+ self.assertFalse(hasattr(ctx, 'ssl_ctx'))
739+ self.assertFalse(hasattr(ctx, 'check_hostname'))
740+
741+ # Test with HTTPS IPv4 URLs:
742+ url = 'https://localhost:6984/'
743+ for env in (url, {'url': url}):
744+ ctx = microfiber.Context(env)
745+ self.assertEqual(ctx.env, {'url': 'https://localhost:6984/'})
746+ self.assertEqual(ctx.basepath, '/')
747+ self.assertEqual(ctx.t,
748+ ('https', 'localhost:6984', '/', '', '', '')
749+ )
750+ self.assertEqual(ctx.url, 'https://localhost:6984/')
751+ self.assertIsInstance(ctx.threadlocal, threading.local)
752+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
753+ self.assertIsNone(ctx.check_hostname)
754+ url = 'https://localhost:6984'
755+ for env in (url, {'url': url}):
756+ ctx = microfiber.Context(env)
757+ self.assertEqual(ctx.env, {'url': 'https://localhost:6984'})
758+ self.assertEqual(ctx.basepath, '/')
759+ self.assertEqual(ctx.t,
760+ ('https', 'localhost:6984', '', '', '', '')
761+ )
762+ self.assertEqual(ctx.url, 'https://localhost:6984/')
763+ self.assertIsInstance(ctx.threadlocal, threading.local)
764+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
765+ self.assertIsNone(ctx.check_hostname)
766+ url = 'https://localhost:6984/bar/'
767+ for env in (url, {'url': url}):
768+ ctx = microfiber.Context(env)
769+ self.assertEqual(ctx.env, {'url': 'https://localhost:6984/bar/'})
770+ self.assertEqual(ctx.basepath, '/bar/')
771+ self.assertEqual(ctx.t,
772+ ('https', 'localhost:6984', '/bar/', '', '', '')
773+ )
774+ self.assertEqual(ctx.url, 'https://localhost:6984/bar/')
775+ self.assertIsInstance(ctx.threadlocal, threading.local)
776+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
777+ self.assertIsNone(ctx.check_hostname)
778+ url = 'https://localhost:6984/bar'
779+ for env in (url, {'url': url}):
780+ ctx = microfiber.Context(env)
781+ self.assertEqual(ctx.env, {'url': 'https://localhost:6984/bar'})
782+ self.assertEqual(ctx.basepath, '/bar/')
783+ self.assertEqual(ctx.t,
784+ ('https', 'localhost:6984', '/bar', '', '', '')
785+ )
786+ self.assertEqual(ctx.url, 'https://localhost:6984/bar/')
787+ self.assertIsInstance(ctx.threadlocal, threading.local)
788+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
789+ self.assertIsNone(ctx.check_hostname)
790+
791+ # Test with HTTPS IPv6 URLs:
792+ url = 'https://[::1]:6984/'
793+ for env in (url, {'url': url}):
794+ ctx = microfiber.Context(env)
795+ self.assertEqual(ctx.env, {'url': 'https://[::1]:6984/'})
796+ self.assertEqual(ctx.basepath, '/')
797+ self.assertEqual(ctx.t,
798+ ('https', '[::1]:6984', '/', '', '', '')
799+ )
800+ self.assertEqual(ctx.url, 'https://[::1]:6984/')
801+ self.assertIsInstance(ctx.threadlocal, threading.local)
802+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
803+ self.assertIsNone(ctx.check_hostname)
804+ url = 'https://[::1]:6984'
805+ for env in (url, {'url': url}):
806+ ctx = microfiber.Context(env)
807+ self.assertEqual(ctx.env, {'url': 'https://[::1]:6984'})
808+ self.assertEqual(ctx.basepath, '/')
809+ self.assertEqual(ctx.t,
810+ ('https', '[::1]:6984', '', '', '', '')
811+ )
812+ self.assertEqual(ctx.url, 'https://[::1]:6984/')
813+ self.assertIsInstance(ctx.threadlocal, threading.local)
814+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
815+ self.assertIsNone(ctx.check_hostname)
816+ url = 'https://[::1]:6984/bar/'
817+ for env in (url, {'url': url}):
818+ ctx = microfiber.Context(env)
819+ self.assertEqual(ctx.env, {'url': 'https://[::1]:6984/bar/'})
820+ self.assertEqual(ctx.basepath, '/bar/')
821+ self.assertEqual(ctx.t,
822+ ('https', '[::1]:6984', '/bar/', '', '', '')
823+ )
824+ self.assertEqual(ctx.url, 'https://[::1]:6984/bar/')
825+ self.assertIsInstance(ctx.threadlocal, threading.local)
826+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
827+ self.assertIsNone(ctx.check_hostname)
828+ url = 'https://[::1]:6984/bar'
829+ for env in (url, {'url': url}):
830+ ctx = microfiber.Context(env)
831+ self.assertEqual(ctx.env, {'url': 'https://[::1]:6984/bar'})
832+ self.assertEqual(ctx.basepath, '/bar/')
833+ self.assertEqual(ctx.t,
834+ ('https', '[::1]:6984', '/bar', '', '', '')
835+ )
836+ self.assertEqual(ctx.url, 'https://[::1]:6984/bar/')
837+ self.assertIsInstance(ctx.threadlocal, threading.local)
838+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
839+ self.assertIsNone(ctx.check_hostname)
840+
841+ # Test with check_hostname=False
842+ env = {
843+ 'url': 'https://127.0.0.1:6984/',
844+ 'ssl': {'check_hostname': False},
845+ }
846+ ctx = microfiber.Context(env)
847+ self.assertEqual(ctx.env,
848+ {
849+ 'url': 'https://127.0.0.1:6984/',
850+ 'ssl': {'check_hostname': False},
851+ }
852+ )
853+ self.assertEqual(ctx.basepath, '/')
854+ self.assertEqual(ctx.t,
855+ ('https', '127.0.0.1:6984', '/', '', '', '')
856+ )
857+ self.assertEqual(ctx.url, 'https://127.0.0.1:6984/')
858+ self.assertIsInstance(ctx.threadlocal, threading.local)
859+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
860+ self.assertIs(ctx.check_hostname, False)
861+ env = {
862+ 'url': 'https://[::1]:6984/',
863+ 'ssl': {'check_hostname': False},
864+ }
865+ ctx = microfiber.Context(env)
866+ self.assertEqual(ctx.env,
867+ {
868+ 'url': 'https://[::1]:6984/',
869+ 'ssl': {'check_hostname': False},
870+ }
871+ )
872+ self.assertEqual(ctx.basepath, '/')
873+ self.assertEqual(ctx.t,
874+ ('https', '[::1]:6984', '/', '', '', '')
875+ )
876+ self.assertEqual(ctx.url, 'https://[::1]:6984/')
877+ self.assertIsInstance(ctx.threadlocal, threading.local)
878+ self.assertIsInstance(ctx.ssl_ctx, ssl.SSLContext)
879+ self.assertIs(ctx.check_hostname, False)
880+
881+ def test_full_url(self):
882+ ctx = microfiber.Context('https://localhost:5003/')
883+ self.assertEqual(
884+ ctx.full_url('/'),
885+ 'https://localhost:5003/'
886+ )
887+ self.assertEqual(
888+ ctx.full_url('/db/doc/att?bar=null&foo=true'),
889+ 'https://localhost:5003/db/doc/att?bar=null&foo=true'
890+ )
891+
892+ ctx = microfiber.Context('https://localhost:5003/mydb/')
893+ self.assertEqual(
894+ ctx.full_url('/'),
895+ 'https://localhost:5003/'
896+ )
897+ self.assertEqual(
898+ ctx.full_url('/db/doc/att?bar=null&foo=true'),
899+ 'https://localhost:5003/db/doc/att?bar=null&foo=true'
900+ )
901+
902+ for url in microfiber.URL_CONSTANTS:
903+ ctx = microfiber.Context(url)
904+ self.assertEqual(ctx.full_url('/'), url)
905+
906+ def test_get_connection(self):
907+ ctx = microfiber.Context(microfiber.HTTP_IPv4_URL)
908+ conn = ctx.get_connection()
909+ self.assertIsInstance(conn, HTTPConnection)
910+ self.assertNotIsInstance(conn, HTTPSConnection)
911+ self.assertEqual(conn.host, '127.0.0.1')
912+ self.assertEqual(conn.port, 5984)
913+
914+ ctx = microfiber.Context(microfiber.HTTP_IPv6_URL)
915+ conn = ctx.get_connection()
916+ self.assertIsInstance(conn, HTTPConnection)
917+ self.assertNotIsInstance(conn, HTTPSConnection)
918+ self.assertEqual(conn.host, '::1')
919+ self.assertEqual(conn.port, 5984)
920+
921+ ctx = microfiber.Context(microfiber.HTTPS_IPv4_URL)
922+ conn = ctx.get_connection()
923+ self.assertIsInstance(conn, HTTPConnection)
924+ self.assertIsInstance(conn, HTTPSConnection)
925+ self.assertEqual(conn.host, '127.0.0.1')
926+ self.assertEqual(conn.port, 6984)
927+ self.assertIs(conn._context, ctx.ssl_ctx)
928+ self.assertIs(conn._check_hostname, True)
929+
930+ ctx = microfiber.Context(microfiber.HTTPS_IPv6_URL)
931+ conn = ctx.get_connection()
932+ self.assertIsInstance(conn, HTTPConnection)
933+ self.assertIsInstance(conn, HTTPSConnection)
934+ self.assertEqual(conn.host, '::1')
935+ self.assertEqual(conn.port, 6984)
936+ self.assertIs(conn._context, ctx.ssl_ctx)
937+ self.assertIs(conn._check_hostname, True)
938+
939+ env = {
940+ 'url': microfiber.HTTPS_IPv4_URL,
941+ 'ssl': {'check_hostname': False},
942+ }
943+ ctx = microfiber.Context(env)
944+ conn = ctx.get_connection()
945+ self.assertIsInstance(conn, HTTPConnection)
946+ self.assertIsInstance(conn, HTTPSConnection)
947+ self.assertEqual(conn.host, '127.0.0.1')
948+ self.assertEqual(conn.port, 6984)
949+ self.assertIs(conn._context, ctx.ssl_ctx)
950+ self.assertIs(conn._check_hostname, False)
951+
952+ env = {
953+ 'url': microfiber.HTTPS_IPv6_URL,
954+ 'ssl': {'check_hostname': False},
955+ }
956+ ctx = microfiber.Context(env)
957+ conn = ctx.get_connection()
958+ self.assertIsInstance(conn, HTTPConnection)
959+ self.assertIsInstance(conn, HTTPSConnection)
960+ self.assertEqual(conn.host, '::1')
961+ self.assertEqual(conn.port, 6984)
962+ self.assertIs(conn._context, ctx.ssl_ctx)
963+ self.assertIs(conn._check_hostname, False)
964+
965+ def test_get_threadlocal_connection(self):
966+ id1 = test_id()
967+ id2 = test_id()
968+
969+ class ContextSubclass(microfiber.Context):
970+ def __init__(self):
971+ self.threadlocal = threading.local()
972+ self._calls = 0
973+
974+ def get_connection(self):
975+ self._calls += 1
976+ return id1
977+
978+ # Test when connection does *not* exist in current thread
979+ ctx = ContextSubclass()
980+ self.assertEqual(ctx.get_threadlocal_connection(), id1)
981+ self.assertEqual(ctx.threadlocal.connection, id1)
982+ self.assertEqual(ctx._calls, 1)
983+ self.assertEqual(ctx.get_threadlocal_connection(), id1)
984+ self.assertEqual(ctx.threadlocal.connection, id1)
985+ self.assertEqual(ctx._calls, 1)
986+ del ctx.threadlocal.connection
987+ self.assertEqual(ctx.get_threadlocal_connection(), id1)
988+ self.assertEqual(ctx.threadlocal.connection, id1)
989+ self.assertEqual(ctx._calls, 2)
990+ self.assertEqual(ctx.get_threadlocal_connection(), id1)
991+ self.assertEqual(ctx.threadlocal.connection, id1)
992+ self.assertEqual(ctx._calls, 2)
993+
994+ # Test when connection does exist in current thread
995+ ctx = ContextSubclass()
996+ ctx.threadlocal.connection = id2
997+ self.assertEqual(ctx.get_threadlocal_connection(), id2)
998+ self.assertEqual(ctx.threadlocal.connection, id2)
999+ self.assertEqual(ctx._calls, 0)
1000+ self.assertEqual(ctx.get_threadlocal_connection(), id2)
1001+ self.assertEqual(ctx.threadlocal.connection, id2)
1002+ self.assertEqual(ctx._calls, 0)
1003+ del ctx.threadlocal.connection
1004+ self.assertEqual(ctx.get_threadlocal_connection(), id1)
1005+ self.assertEqual(ctx.threadlocal.connection, id1)
1006+ self.assertEqual(ctx._calls, 1)
1007+ self.assertEqual(ctx.get_threadlocal_connection(), id1)
1008+ self.assertEqual(ctx.threadlocal.connection, id1)
1009+ self.assertEqual(ctx._calls, 1)
1010+
1011+ # Sanity check with the original class:
1012+ ctx = microfiber.Context(microfiber.HTTPS_IPv6_URL)
1013+ conn = ctx.get_threadlocal_connection()
1014+ self.assertIs(conn, ctx.threadlocal.connection)
1015+ self.assertIsInstance(conn, HTTPConnection)
1016+ self.assertIsInstance(conn, HTTPSConnection)
1017+ self.assertEqual(conn.host, '::1')
1018+ self.assertEqual(conn.port, 6984)
1019+ self.assertIs(conn._context, ctx.ssl_ctx)
1020+ self.assertIs(conn._check_hostname, True)
1021+ self.assertIs(ctx.get_threadlocal_connection(), conn)
1022+ self.assertIs(conn, ctx.threadlocal.connection)
1023+
1024+ def test_get_auth_headers(self):
1025+ method = 'GET'
1026+ path = '/photos'
1027+ query = (('file', 'vacation.jpg'), ('size', 'original'))
1028+ testing = ('1191242096', 'kllo9940pd9333jh')
1029+
1030+ # Test with no-auth (open):
1031+ env = {
1032+ 'url': 'http://photos.example.net/',
1033+ }
1034+ ctx = microfiber.Context(env)
1035+ self.assertEqual(
1036+ ctx.get_auth_headers(method, path, query, testing),
1037+ {}
1038+ )
1039+
1040+ # Test with basic auth:
1041+ env = {
1042+ 'url': 'http://photos.example.net/',
1043+ 'basic': {'username': 'Aladdin', 'password': 'open sesame'},
1044+ }
1045+ ctx = microfiber.Context(env)
1046+ self.assertEqual(
1047+ ctx.get_auth_headers(method, path, query, testing),
1048+ {'Authorization': 'Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ=='}
1049+ )
1050+
1051+ # Test with oauth:
1052+ env = {
1053+ 'url': 'http://photos.example.net/',
1054+ 'oauth': dict(SAMPLE_OAUTH_TOKENS),
1055+ }
1056+ ctx = microfiber.Context(env)
1057+ self.assertEqual(
1058+ ctx.get_auth_headers(method, path, query, testing),
1059+ {'Authorization': SAMPLE_OAUTH_AUTHORIZATION},
1060+ )
1061+
1062+ # Make sure oauth overrides basic
1063+ env = {
1064+ 'url': 'http://photos.example.net/',
1065+ 'oauth': dict(SAMPLE_OAUTH_TOKENS),
1066+ 'basic': {'username': 'Aladdin', 'password': 'open sesame'},
1067+ }
1068+ ctx = microfiber.Context(env)
1069+ self.assertEqual(
1070+ ctx.get_auth_headers(method, path, query, testing),
1071+ {'Authorization': SAMPLE_OAUTH_AUTHORIZATION},
1072+ )
1073+
1074+
1075+class TestCouchBase(TestCase):
1076+ def test_init(self):
1077+ # Supply neither *env* nor *ctx*:
1078 inst = microfiber.CouchBase()
1079- self.assertIsInstance(inst._threadlocal, threading.local)
1080- value = random_id()
1081- inst._threadlocal.conn = value
1082- self.assertEqual(inst.conn, value)
1083- delattr(inst._threadlocal, 'conn')
1084- self.assertIsInstance(inst.conn, HTTPConnection)
1085+ self.assertIsInstance(inst.ctx, microfiber.Context)
1086+ self.assertEqual(inst.env, {'url': microfiber.HTTP_IPv4_URL})
1087+ self.assertIs(inst.env, inst.ctx.env)
1088+ self.assertEqual(inst.basepath, '/')
1089+ self.assertIs(inst.basepath, inst.ctx.basepath)
1090+ self.assertEqual(inst.url, microfiber.HTTP_IPv4_URL)
1091+ self.assertIs(inst.url, inst.ctx.url)
1092+
1093+ # Supply *env*:
1094+ env = {'url': microfiber.HTTPS_IPv6_URL}
1095+ inst = microfiber.CouchBase(env=env)
1096+ self.assertIsInstance(inst.ctx, microfiber.Context)
1097+ self.assertEqual(inst.env, {'url': microfiber.HTTPS_IPv6_URL})
1098+ self.assertIs(inst.env, inst.ctx.env)
1099+ self.assertIs(inst.env, env)
1100+ self.assertEqual(inst.basepath, '/')
1101+ self.assertIs(inst.basepath, inst.ctx.basepath)
1102+ self.assertEqual(inst.url, microfiber.HTTPS_IPv6_URL)
1103+ self.assertIs(inst.url, inst.ctx.url)
1104+
1105+ # Supply *ctx*:
1106+ url = 'http://example.com/foo/'
1107+ ctx = microfiber.Context(url)
1108+ inst = microfiber.CouchBase(ctx=ctx)
1109+ self.assertIsInstance(inst.ctx, microfiber.Context)
1110+ self.assertIs(inst.ctx, ctx)
1111+ self.assertEqual(inst.env, {'url': url})
1112+ self.assertIs(inst.env, inst.ctx.env)
1113+ self.assertEqual(inst.basepath, '/foo/')
1114+ self.assertIs(inst.basepath, inst.ctx.basepath)
1115+ self.assertEqual(inst.url, url)
1116+ self.assertIs(inst.url, inst.ctx.url)
1117
1118 def test_full_url(self):
1119- inst = self.klass('https://localhost:5003/')
1120+ inst = microfiber.CouchBase('https://localhost:5003/')
1121 self.assertEqual(
1122 inst._full_url('/'),
1123 'https://localhost:5003/'
1124@@ -1057,7 +1545,7 @@
1125 'https://localhost:5003/db/doc/att?bar=null&foo=true'
1126 )
1127
1128- inst = self.klass('http://localhost:5003/mydb/')
1129+ inst = microfiber.CouchBase('http://localhost:5003/mydb/')
1130 self.assertEqual(
1131 inst._full_url('/'),
1132 'http://localhost:5003/'
1133@@ -1069,65 +1557,91 @@
1134
1135
1136 class TestServer(TestCase):
1137- klass = microfiber.Server
1138-
1139 def test_init(self):
1140- inst = self.klass()
1141- self.assertEqual(inst.url, 'http://localhost:5984/')
1142- self.assertEqual(inst.basepath, '/')
1143- self.assertIsInstance(inst.conn, HTTPConnection)
1144- self.assertIs(inst.Conn, HTTPConnection)
1145- self.assertNotIsInstance(inst.conn, HTTPSConnection)
1146-
1147- inst = self.klass('https://localhost:6000')
1148- self.assertEqual(inst.url, 'https://localhost:6000/')
1149- self.assertEqual(inst.basepath, '/')
1150- self.assertIsInstance(inst.conn, HTTPSConnection)
1151- self.assertIs(inst.Conn, HTTPSConnection)
1152-
1153- inst = self.klass('http://example.com/foo')
1154- self.assertEqual(inst.url, 'http://example.com/foo/')
1155+ # Supply neither *env* nor *ctx*:
1156+ inst = microfiber.Server()
1157+ self.assertIsInstance(inst.ctx, microfiber.Context)
1158+ self.assertEqual(inst.env, {'url': microfiber.HTTP_IPv4_URL})
1159+ self.assertIs(inst.env, inst.ctx.env)
1160+ self.assertEqual(inst.basepath, '/')
1161+ self.assertIs(inst.basepath, inst.ctx.basepath)
1162+ self.assertEqual(inst.url, microfiber.HTTP_IPv4_URL)
1163+ self.assertIs(inst.url, inst.ctx.url)
1164+
1165+ # Supply *env*:
1166+ env = {'url': microfiber.HTTPS_IPv6_URL}
1167+ inst = microfiber.Server(env=env)
1168+ self.assertIsInstance(inst.ctx, microfiber.Context)
1169+ self.assertEqual(inst.env, {'url': microfiber.HTTPS_IPv6_URL})
1170+ self.assertIs(inst.env, inst.ctx.env)
1171+ self.assertIs(inst.env, env)
1172+ self.assertEqual(inst.basepath, '/')
1173+ self.assertIs(inst.basepath, inst.ctx.basepath)
1174+ self.assertEqual(inst.url, microfiber.HTTPS_IPv6_URL)
1175+ self.assertIs(inst.url, inst.ctx.url)
1176+
1177+ # Supply *ctx*:
1178+ url = 'http://example.com/foo/'
1179+ ctx = microfiber.Context(url)
1180+ inst = microfiber.Server(ctx=ctx)
1181+ self.assertIsInstance(inst.ctx, microfiber.Context)
1182+ self.assertIs(inst.ctx, ctx)
1183+ self.assertEqual(inst.env, {'url': url})
1184+ self.assertIs(inst.env, inst.ctx.env)
1185 self.assertEqual(inst.basepath, '/foo/')
1186- self.assertIsInstance(inst.conn, HTTPConnection)
1187- self.assertIs(inst.Conn, HTTPConnection)
1188- self.assertNotIsInstance(inst.conn, HTTPSConnection)
1189-
1190- inst = self.klass('https://example.com/bar')
1191- self.assertEqual(inst.url, 'https://example.com/bar/')
1192- self.assertEqual(inst.basepath, '/bar/')
1193- self.assertIsInstance(inst.conn, HTTPSConnection)
1194- self.assertIs(inst.Conn, HTTPSConnection)
1195-
1196- inst = self.klass({'oauth': 'bar'})
1197- self.assertEqual(inst._oauth, 'bar')
1198+ self.assertIs(inst.basepath, inst.ctx.basepath)
1199+ self.assertEqual(inst.url, url)
1200+ self.assertIs(inst.url, inst.ctx.url)
1201
1202 def test_repr(self):
1203- inst = self.klass('http://localhost:5001/')
1204- self.assertEqual(repr(inst), "Server('http://localhost:5001/')")
1205-
1206- inst = self.klass('http://localhost:5002')
1207- self.assertEqual(repr(inst), "Server('http://localhost:5002/')")
1208-
1209- inst = self.klass('https://localhost:5003/')
1210- self.assertEqual(repr(inst), "Server('https://localhost:5003/')")
1211-
1212- inst = self.klass('https://localhost:5004')
1213- self.assertEqual(repr(inst), "Server('https://localhost:5004/')")
1214+ # Use a subclass to make sure only Server.url factors into __repr__():
1215+ class ServerSubclass(microfiber.Server):
1216+ def __init__(self):
1217+ pass
1218+
1219+ inst = ServerSubclass()
1220+ inst.url = microfiber.HTTP_IPv4_URL
1221+ self.assertEqual(repr(inst),
1222+ "ServerSubclass('http://127.0.0.1:5984/')"
1223+ )
1224+ inst.url = microfiber.HTTPS_IPv4_URL
1225+ self.assertEqual(repr(inst),
1226+ "ServerSubclass('https://127.0.0.1:6984/')"
1227+ )
1228+ inst.url = microfiber.HTTP_IPv6_URL
1229+ self.assertEqual(repr(inst),
1230+ "ServerSubclass('http://[::1]:5984/')"
1231+ )
1232+ inst.url = microfiber.HTTPS_IPv6_URL
1233+ self.assertEqual(repr(inst),
1234+ "ServerSubclass('https://[::1]:6984/')"
1235+ )
1236+
1237+ # Sanity check with original class
1238+ inst = microfiber.Server()
1239+ self.assertEqual(repr(inst),
1240+ "Server('http://127.0.0.1:5984/')"
1241+ )
1242+ inst = microfiber.Server(microfiber.HTTPS_IPv6_URL)
1243+ self.assertEqual(repr(inst),
1244+ "Server('https://[::1]:6984/')"
1245+ )
1246
1247 def test_database(self):
1248- s = microfiber.Server()
1249- db = s.database('mydb')
1250- self.assertIsInstance(db, microfiber.Database)
1251- self.assertIsNone(db._basic)
1252- self.assertIsNone(db._oauth)
1253-
1254- s = microfiber.Server({'basic': 'foo', 'oauth': 'bar'})
1255- self.assertEqual(s._basic, 'foo')
1256- self.assertEqual(s._oauth, 'bar')
1257- db = s.database('mydb')
1258- self.assertIsInstance(db, microfiber.Database)
1259- self.assertEqual(s._basic, 'foo')
1260- self.assertEqual(s._oauth, 'bar')
1261+ server = microfiber.Server()
1262+ db = server.database('mydb')
1263+ self.assertIsInstance(db, microfiber.Database)
1264+ self.assertIs(db.ctx, server.ctx)
1265+ self.assertEqual(db.name, 'mydb')
1266+ self.assertEqual(db.basepath, '/mydb/')
1267+
1268+ server = microfiber.Server('http://example.com/foo/')
1269+ db = server.database('mydb')
1270+ self.assertIsInstance(db, microfiber.Database)
1271+ self.assertIs(db.ctx, server.ctx)
1272+ self.assertEqual(db.name, 'mydb')
1273+ self.assertEqual(db.basepath, '/foo/mydb/')
1274+ self.assertEqual(db.url, 'http://example.com/foo/')
1275
1276
1277 class TestDatabase(TestCase):
1278@@ -1136,7 +1650,7 @@
1279 def test_init(self):
1280 inst = self.klass('foo')
1281 self.assertEqual(inst.name, 'foo')
1282- self.assertEqual(inst.url, 'http://localhost:5984/')
1283+ self.assertEqual(inst.url, 'http://127.0.0.1:5984/')
1284 self.assertEqual(inst.basepath, '/foo/')
1285
1286 inst = self.klass('baz', 'https://example.com/bar')
1287@@ -1148,7 +1662,7 @@
1288 inst = self.klass('dmedia')
1289 self.assertEqual(
1290 repr(inst),
1291- "Database('dmedia', 'http://localhost:5984/')"
1292+ "Database('dmedia', 'http://127.0.0.1:5984/')"
1293 )
1294
1295 inst = self.klass('novacut', 'https://localhost:5004/')
1296@@ -1159,26 +1673,17 @@
1297
1298 def test_server(self):
1299 db = microfiber.Database('mydb')
1300- self.assertIsNone(db._basic)
1301- self.assertIsNone(db._oauth)
1302- s = db.server()
1303- self.assertIsInstance(s, microfiber.Server)
1304- self.assertEqual(s.url, 'http://localhost:5984/')
1305- self.assertEqual(s.basepath, '/')
1306- self.assertIsNone(s._basic)
1307- self.assertIsNone(s._oauth)
1308+ server = db.server()
1309+ self.assertIsInstance(server, microfiber.Server)
1310+ self.assertIs(server.ctx, db.ctx)
1311+ self.assertEqual(server.basepath, '/')
1312
1313- db = microfiber.Database('mydb',
1314- {'url': 'https://example.com/stuff', 'basic': 'foo', 'oauth': 'bar'}
1315- )
1316- self.assertEqual(db._basic, 'foo')
1317- self.assertEqual(db._oauth, 'bar')
1318- s = db.server()
1319- self.assertIsInstance(s, microfiber.Server)
1320- self.assertEqual(s.url, 'https://example.com/stuff/')
1321- self.assertEqual(s.basepath, '/stuff/')
1322- self.assertEqual(s._basic, 'foo')
1323- self.assertEqual(s._oauth, 'bar')
1324+ db = microfiber.Database('mydb', {'url': 'https://example.com/stuff'})
1325+ server = db.server()
1326+ self.assertIsInstance(server, microfiber.Server)
1327+ self.assertIs(server.ctx, db.ctx)
1328+ self.assertEqual(server.url, 'https://example.com/stuff/')
1329+ self.assertEqual(server.basepath, '/stuff/')
1330
1331 def test_view(self):
1332 class Mock(microfiber.Database):
1333@@ -1213,15 +1718,45 @@
1334 self.assertEqual(db._options, {'reduce': True, 'include_docs': True})
1335
1336
1337-class ReplicationTestCase(TestCase):
1338+class LiveTestCase(TestCase):
1339+ """
1340+ Base class for tests that can be skipped via the --no-live option.
1341+
1342+ When working on code whose tests don't need a live CouchDB instance, its
1343+ annoying to wait for the slow live tests to run. You can skip the live
1344+ tests like this::
1345+
1346+ ./setup.py test --no-live
1347+
1348+ Sub-classes should call ``super().setUp()`` first thing in their
1349+ ``setUp()`` methods.
1350+ """
1351+
1352 def setUp(self):
1353 if os.environ.get('MICROFIBER_TEST_NO_LIVE') == 'true':
1354- self.skipTest('called with --no-live')
1355- if usercouch is None:
1356- self.skipTest('`usercouch` not installed')
1357- self.tmp1 = usercouch.misc.TempCouch()
1358+ self.skipTest('run with --no-live')
1359+
1360+
1361+class CouchTestCase(LiveTestCase):
1362+ db = 'test_microfiber'
1363+
1364+ def setUp(self):
1365+ super().setUp()
1366+ self.auth = os.environ.get('MICROFIBER_TEST_AUTH', 'basic')
1367+ self.tmpcouch = TempCouch()
1368+ self.env = self.tmpcouch.bootstrap(self.auth)
1369+
1370+ def tearDown(self):
1371+ self.tmpcouch = None
1372+ self.env = None
1373+
1374+
1375+class ReplicationTestCase(LiveTestCase):
1376+ def setUp(self):
1377+ super().setUp()
1378+ self.tmp1 = TempCouch()
1379 self.env1 = self.tmp1.bootstrap()
1380- self.tmp2 = usercouch.misc.TempCouch()
1381+ self.tmp2 = TempCouch()
1382 self.env2 = self.tmp2.bootstrap()
1383
1384 def tearDown(self):
1385@@ -1305,24 +1840,7 @@
1386 self.assertEqual(s2.get(name2, doc['_id']), doc)
1387
1388
1389-class LiveTestCase(TestCase):
1390- db = 'test_microfiber'
1391-
1392- def setUp(self):
1393- if os.environ.get('MICROFIBER_TEST_NO_LIVE') == 'true':
1394- self.skipTest('called with --no-live')
1395- if usercouch is None:
1396- self.skipTest('`usercouch` not installed')
1397- self.auth = os.environ.get('MICROFIBER_TEST_AUTH', 'basic')
1398- self.tmpcouch = usercouch.misc.TempCouch()
1399- self.env = self.tmpcouch.bootstrap(self.auth)
1400-
1401- def tearDown(self):
1402- self.tmpcouch = None
1403- self.env = None
1404-
1405-
1406-class TestFakeList(LiveTestCase):
1407+class TestFakeList(CouchTestCase):
1408 def test_init(self):
1409 db = microfiber.Database('foo', self.env)
1410 self.assertTrue(db.ensure())
1411@@ -1368,7 +1886,7 @@
1412 self.assertEqual(list(fake), orig)
1413
1414
1415-class TestCouchBaseLive(LiveTestCase):
1416+class TestCouchBaseLive(CouchTestCase):
1417 klass = microfiber.CouchBase
1418
1419 def test_bad_status_line(self):
1420@@ -1606,10 +2124,81 @@
1421 # Delete the database
1422 self.assertEqual(inst.delete(self.db), {'ok': True})
1423 self.assertRaises(NotFound, inst.delete, self.db)
1424- self.assertRaises(NotFound, inst.get, self.db)
1425-
1426-
1427-class TestDatabaseLive(LiveTestCase):
1428+ self.assertRaises(NotFound, inst.get, self.db)
1429+
1430+
1431+class TestPermutations(LiveTestCase):
1432+ """
1433+ Test `CouchBase._request()` over all key *env* permutations.
1434+ """
1435+
1436+ # FIXME: For some reason OAuth isn't working with IPv6, perhap
1437+ # server and client aren't using same canonical URL when signing?
1438+
1439+ bind_addresses = ('127.0.0.1', '::1')
1440+ auths = ('open', 'basic', 'oauth')
1441+
1442+ def check_with_bad_auth(self, env, auth):
1443+ """
1444+ Sanity check to make sure UserCouch configured CouchDB as expected.
1445+ """
1446+ if auth == 'basic':
1447+ bad = deepcopy(env)
1448+ bad['basic']['password'] = random_id()
1449+ uc = microfiber.CouchBase(bad)
1450+ with self.assertRaises(microfiber.Unauthorized):
1451+ uc.get()
1452+ elif auth == 'oauth':
1453+ bad = deepcopy(env)
1454+ bad['oauth']['token_secret'] = random_id()
1455+ uc = microfiber.CouchBase(bad)
1456+ with self.assertRaises(microfiber.Unauthorized):
1457+ uc.get()
1458+
1459+ def test_http(self):
1460+ for bind_address in self.bind_addresses:
1461+ for auth in self.auths:
1462+ if auth == 'oauth' and bind_address == '::1':
1463+ continue
1464+ tmpcouch = TempCouch()
1465+ env = tmpcouch.bootstrap(auth, {'bind_address': bind_address})
1466+ uc = microfiber.CouchBase(env)
1467+ self.assertEqual(uc.get()['couchdb'], 'Welcome')
1468+ self.check_with_bad_auth(env, auth)
1469+
1470+ def test_https(self):
1471+ pki = TempPKI()
1472+ for bind_address in self.bind_addresses:
1473+ for auth in self.auths:
1474+ if auth == 'oauth' and bind_address == '::1':
1475+ continue
1476+ config = {
1477+ 'bind_address': bind_address,
1478+ 'ssl': pki.get_server_config()
1479+ }
1480+ tmpcouch = TempCouch()
1481+ env = tmpcouch.bootstrap(auth, config)['x_env_ssl']
1482+ env['ssl'] = pki.get_client_config()
1483+ uc = microfiber.CouchBase(env)
1484+ self.assertEqual(uc.get()['couchdb'], 'Welcome')
1485+ self.check_with_bad_auth(env, auth)
1486+
1487+ # Make sure things fail without ca_file
1488+ bad = deepcopy(env)
1489+ del bad['ssl']['ca_file']
1490+ uc = microfiber.CouchBase(bad)
1491+ with self.assertRaises(ssl.SSLError) as cm:
1492+ uc.get()
1493+
1494+ # Make sure things fail without {'check_hostname': False}
1495+ bad = deepcopy(env)
1496+ del bad['ssl']['check_hostname']
1497+ uc = microfiber.CouchBase(bad)
1498+ with self.assertRaises(ssl.CertificateError) as cm:
1499+ uc.get()
1500+
1501+
1502+class TestDatabaseLive(CouchTestCase):
1503 klass = microfiber.Database
1504
1505 def test_ensure(self):
1506@@ -2047,7 +2636,7 @@
1507 db.save_many(docs)
1508
1509 # Test with .json
1510- dst = path.join(self.tmpcouch.paths.bzr, 'foo.json')
1511+ dst = path.join(self.tmpcouch.paths.dump, 'foo.json')
1512 db.dump(dst)
1513 self.assertEqual(open(dst, 'r').read(), docs_s)
1514 self.assertEqual(
1515@@ -2056,7 +2645,7 @@
1516 )
1517
1518 # Test with .json.gz
1519- dst = path.join(self.tmpcouch.paths.bzr, 'foo.json.gz')
1520+ dst = path.join(self.tmpcouch.paths.dump, 'foo.json.gz')
1521 db.dump(dst)
1522 gz_checksum = md5(open(dst, 'rb').read()).hexdigest()
1523 self.assertEqual(
1524@@ -2073,7 +2662,7 @@
1525 )
1526
1527 # Test that filename doesn't change gz_checksum
1528- dst = path.join(self.tmpcouch.paths.bzr, 'bar.json.gz')
1529+ dst = path.join(self.tmpcouch.paths.dump, 'bar.json.gz')
1530 db.dump(dst)
1531 self.assertEqual(
1532 md5(open(dst, 'rb').read()).hexdigest(),
1533@@ -2081,7 +2670,7 @@
1534 )
1535
1536 # Make sure .JSON.GZ also works, that case is ignored
1537- dst = path.join(self.tmpcouch.paths.bzr, 'FOO.JSON.GZ')
1538+ dst = path.join(self.tmpcouch.paths.dump, 'FOO.JSON.GZ')
1539 db.dump(dst)
1540 self.assertEqual(
1541 md5(open(dst, 'rb').read()).hexdigest(),

Subscribers

People subscribed via source and target branches