Merge lp:~jderose/dmedia/peering into lp:dmedia

[Jason Gerard DeRose]

I should have done the httpd.py module in a separate merge, but this is a good point to merge this before it becomes even more of a mega merge.

The important thing here is the new httpd.py module and its tests. There is also sundry work in progress on the secure peering protocol and the new SSL based machine and user identity stuff.

httpd.py contains a tiny WSGI server designed to do just enough for our needs. In terms of why I wrote one, there aren't a lot of options for our Python3 + IPv6 + SSL requirements. gunicorn was my first thought, be it doesn't have a stable Python3 release yet.

And the server I wrote is less than 400 lines of Python, so who cares. By only supporting exactly the features we need, we have a lot less code to review for security issues.

The server already supports everything we need for Dmedia file transfer. There might be a few things we need to add once I dive into the details of exactly how the CouchDB replicator behaves. In particular, we might need to add support for chunked-encoding, and perhaps support for "100 Continue".

The file transfer logic will remain on the WSGI app side of things, as it currently is. Likewise, the logic for proxying requests to CouchDB will be in the WSGI app. So it will be easy for us to swap out to a different WSGI server should another seems like a better fit.

[lp:~jderose/dmedia/peering]

513. By Jason Gerard DeRose on 2012-09-30

WSGIError is now raised on duplicate headers; added WSGIError class docstring explaining its use

514. By Jason Gerard DeRose on 2012-09-30

fixed 'preample' typo, thanks David

515. By Jason Gerard DeRose on 2012-09-30

build_server_ssl_context() now ors options with ssl.OP_NO_COMPRESSION as a safety against CRIME type attacks

516. By Jason Gerard DeRose on 2012-09-30

Added comment about CRIME-like attacks

517. By Jason Gerard DeRose on 2012-09-30

echo_app() now returns md5 of request body as 'echo.content_md5'

518. By Jason Gerard DeRose on 2012-10-01

parse_request() and parse_header() now check for internal CR

519. By Jason Gerard DeRose on 2012-10-01

Small aethetic cleanup

520. By Jason Gerard DeRose on 2012-10-01

Made things simpler by always being threaded

521. By Jason Gerard DeRose on 2012-10-01

benchmark now includes a small request body to be more realistic

522. By Jason Gerard DeRose on 2012-10-01

So we can use tighter Apparmor containment, don't use socket.getfqdn() for SERVER_NAME

523. By Jason Gerard DeRose on 2012-10-01

Added detailed live tests using the echo_app

524. By Jason Gerard DeRose on 2012-10-01

base environ now includes Apache-style {'SSL_COMPRESS_METHOD': 'NULL'}

525. By Jason Gerard DeRose on 2012-10-01

small cleanup

[Jason Gerard DeRose (jderose)]

I'm gonna self-approve this so we can get this through the daily builds and then I do the remaining work in smaller pieces.

Note that none of this functionality is turned on yet by default, so this is safe to merge to trunk, despite parts of it still being works in progress.

review: Approve

[lp:~jderose/dmedia/peering]

526. By Jason Gerard DeRose on 2012-10-01

For now, revert changes to avahi.py so oauth-based peering still works

[Jason Gerard DeRose (jderose)]

review: Approve