Merge lp:~javier.collado/utah/url-doctest into lp:utah

[Javier Collado]

This branch adds documentation and doctests to the utah.url module.

[Max Brustkern (nuclearbob)]

I don't see an ellipsis that would match the +ELLIPSIS directive on line 209 of the merge, so I wonder if that directive is needed.

Other than that, it looks reasonable to me.

review: Needs Information

[lp:~javier.collado/utah/url-doctest]

759. By Javier Collado on 2012-11-27

Removed unnecessary ELLIPSIS

760. By Javier Collado on 2012-11-27

Added missing import

Nosetest works fine without the import, but 'make doctest' in the documentation
directory doesn't.

[Javier Collado (javier.collado)]

You're right, I had a more complex example to show launchpad URLs that
didn't work fine, so I decided to go for something simpler for the
documentation and forgot to remove the ELLIPSIS.

Aside from that, I've added an import that is needed to run the test cases
from the `docs` directory with `make doctest`.

Since it looks good to you, I'm merging right away.

review: Approve

=== modified file 'docs/source/reference.rst'
--- docs/source/reference.rst	2012-11-08 19:36:29 +0000
+++ docs/source/reference.rst	2012-11-27 10:01:20 +0000
@@ -41,8 +41,12 @@
 .. automodule:: utah.timeout
    :members:
 
+``utah.url``
+----------------
+
 .. automodule:: utah.url
    :members:
+   :member-order: bysource
 
 ``utah.client``
 ===============

=== modified file 'utah/exceptions.py'
--- utah/exceptions.py	2012-08-22 15:20:25 +0000
+++ utah/exceptions.py	2012-11-27 10:01:20 +0000
@@ -14,15 +14,3 @@
         except KeyError:
             self.retry = False
         super(UTAHException, self).__init__(*args, **kw)
-
-
-class URLNotFound(UTAHException):
-    """
-    Exception raised when a URL isn't found
-    """
-
-
-class URLNotReadable(UTAHException):
-    """
-    Exception raised when a URL isn't readable
-    """

=== modified file 'utah/url.py'
--- utah/url.py	2012-10-11 15:11:34 +0000
+++ utah/url.py	2012-11-27 10:01:20 +0000
@@ -1,5 +1,8 @@
-"""
-URL tools
+r"""This module provides the classes/functions needed to:
+
+ - Check that a URL is valid and readable
+ - Use a url type in an `argparse.ArgumentParser` object
+
 """
 import os
 import urllib
@@ -13,26 +16,71 @@
 import bzrlib.plugin
 import bzrlib.errors
 
-from utah.exceptions import URLNotFound, URLNotReadable
+from utah.exceptions import UTAHException
 
 
 # Inspired by: http://stackoverflow.com/a/2070916/183066
 class HeadRequest(urllib2.Request):
-    """
-    Request that sends HEAD method instead of GET
-    """
+
+    """A request that sends HEAD method instead of GET.
+
+    .. seealso:: :class:`URLChecker`
+
+    """
+
     def get_method(self):
+        """Return Method used to get URL.
+
+        :returns: 'HEAD'
+        :rtype: str
+
+        """
+
         return 'HEAD'
 
 
 class URLChecker(urllib.URLopener):
-    """
-    Check that URL exists withou opening it
-    or downloading its contents
-    """
-    def open_http(self, url, data=None):
-        """
-        Check if http URL exists
+
+    """An opener to checks a URL is valid and readable.
+
+    To use it, create an object instance and call the `open`
+    method passing the url to be checked as argument.
+
+    """
+
+    def open_http(self, url):
+        """Check if http URL exists and is readable.
+
+        The check is performed by sending an HTTP HEAD
+        request, waiting for the response and checking that
+        the code is 200 OK.
+
+        If a redirect response is received, the URL will still
+        be reported as working fine, but the underlying
+        implementation will use an HTTP GET method instead, so
+        it won't be as efficient as in the standard case.
+
+        :param url: The HTTP URL to be checked
+        :type url: `basestring`
+        :returns: The url passed as argument when it's valid and readable.
+        :rtype: `basestring`
+        :throws URLNotFound:
+            When there's a problem opening the URL or isn't found.
+
+        :Example:
+
+        >>> from utah.url import URLChecker
+        >>> opener = URLChecker()
+        >>> opener.open('http://www.ubuntu.com')
+        'http://www.ubuntu.com'
+
+        .. note::
+
+            This method is called by the `open` method when the URL protocol is
+            http, so it's not expected to be called directly.
+
+        .. seealso:: :meth:`open_local_file`, :class:`URLNotFound`
+
         """
         # This is redundant becuase urllib2 will call urllib
         # under the hood, but makes code easy to read.
@@ -52,8 +100,39 @@
         return url
 
     def open_local_file(self, url):
-        """
-        Check if local file exists
+        """Check if local file exists.
+
+        :param url: The file URL to be checked
+        :type url: `basestring`
+        :returns: The path to the file if it was found and readable.
+
+            .. note::
+                The returned value is a path, not a URL, so it
+                can be used to open the file the same way as
+                any other files.
+        :rtype: `basestring`
+        :throws URLNotFound:
+            when the path to the file doesn't exist.
+        :throws URLNotReadable:
+            when the user doesn't have read permissions to open the file.
+
+        :Example:
+
+        >>> import tempfile
+        >>> with tempfile.NamedTemporaryFile() as f:  # doctest: +ELLIPSIS
+        ...     opener = URLChecker()
+        ...     opener.open(f.name)
+        '/tmp/tmp...'
+
+        .. note::
+
+            This method is called by the `open` method when the URL protocol is
+            file, so it's not expected to be called directly.
+
+        .. seealso::
+
+            :meth:`open_http`, :class:`URLNotFound`, :class:`URLNotReadable`
+
         """
         # Based on urllib.URLopener.open_local_file implementation
         _host, filename = urllib.splithost(url)
@@ -69,8 +148,37 @@
 
 
 def url_argument(url):
-    """
-    Check URL argument in argparse.ArgumentParser
+    """URL argument to be used in an `argparse.ArgumentParser` object.
+
+    :param url: URL as passed to the parser object.
+
+        .. note::
+
+            The URL passed as argument can be a launchpad URL. In that case,
+            the file pointed by the URL will be downloaded as when using `bzr
+            export` and the returned value is the path to the downloaded file.
+
+    :type url: `basestring`
+    :returns: URL or path to local file
+    :rtype: `basestring`
+    :throws argparse.ArgumentTypeError:
+
+        when the URL is invalid or unreadable. In any case, the error message
+        will provide information to be displayed by the
+        `argparse.ArgumentParser` object in the command line.
+
+    :Example:
+
+    >>> from utah.url import url_argument
+    >>> import argparse
+    >>> parser = argparse.ArgumentParser()
+    >>> parser.add_argument('url', type=url_argument)  # doctest: +ELLIPSIS
+    _StoreAction(... dest='url', ...)
+    >>> parser.parse_args(['http://www.ubuntu.com'])
+    Namespace(url='http://www.ubuntu.com')
+
+    .. seealso:: :class:`URLChecker`
+
     """
     parse_result = urlparse(url)
     if parse_result.scheme in ('', 'file', 'http', 'https'):
@@ -87,6 +195,7 @@
         bzr_logger.addHandler(logging.NullHandler())
         bzrlib.plugin.load_plugins()  # Enable launchpad URLs in bazaar
         cmd = bzrlib.builtins.cmd_export()
+        assert cmd is not None
         tmp_dir = tempfile.mkdtemp(prefix='utah_')
         try:
             cmd.run(tmp_dir, url)
@@ -104,3 +213,44 @@
                                 .format(parse_result.scheme, url))
 
     return full_url
+
+
+class URLNotFound(UTAHException):
+
+    """Exception raised when a URL isn't found.
+
+    :Example:
+
+    >>> opener = URLChecker()
+    >>> opener.open('http://localhost/invalid_url')
+    Traceback (most recent call last):
+      ...
+    URLNotFound: http://localhost/invalid_url
+    >>> opener.open('file:///tmp/invalid_url')
+    Traceback (most recent call last):
+      ...
+    URLNotFound: /tmp/invalid_url
+
+    .. seealso:: :class:`URLChecker`
+
+    """
+
+
+class URLNotReadable(UTAHException):
+
+    """Exception raised when a URL isn't readable.
+
+    :Example:
+
+    >>> import os
+    >>> with tempfile.NamedTemporaryFile() as f:  # doctest: +ELLIPSIS
+    ...     os.chmod(f.name, 0000)
+    ...     opener = URLChecker()
+    ...     opener.open(f.name)
+    Traceback (most recent call last):
+      ...
+    URLNotReadable: /tmp/tmp...
+
+    .. seealso:: :class:`URLChecker`
+
+    """