Merge lp:~hernejj/ubuntu-qa-tools/better_hugday into lp:ubuntu-qa-tools

[Jason J. Herne]

Adds code to do the following:
- Supported browsers: Firefox, Epiphany, Seamonkey, Google Chrome, Konqueror
- Automatic cookie detection for all supported browsers.
- User no longer has to close the browser to find cookies.
- Cookie file format automatically detected when --cookie is given
- User can specify a browser (--browser) and cookie is automatically found within that browser's cookie files.
- Some simple code cleanup and extra commenting in hugday tool.

I've been working on some improvements to the hugday tool (part of the ubuntu-qa-tools) that should make hugday easier to use. Making hugday easier to use will lower the "barrier of entry" for new comers who are interested in participating in the Ubuntu Bug Days. And anything we can do to help people help Ubuntu is a good idea in my book ;).

I've added code to allow for more robust cookie handling by the "hugday init" sub-command. Here is a list of features currently implemented in my branch:
- Supported browsers: Firefox, Epiphany, Seamonkey, Google Chrome, Konqueror
- Automatic cookie detection for all supported browsers.
- User no longer has to close the browser to find cookies.
- Cookie file format automatically detected when --cookie is given
- User can specify a browser (--browser) and cookie is automatically found within that browser's cookie files.
- Some simple code cleanup and extra commenting in hugday tool.

If this is something that bug-control is interested in I'd be happy to have these features merged back into the official ubuntu-qa-tools-branch. Although I must admit I'm not sure what the process is for officially requesting such a merge.

[Brian Murray (brian-murray)]

This sounds really exciting! You have followed the proper process for requesting such a merge but unfortunately I don't think we can get this into Lucid since we are past feature freeze. I'm requesting that the original author look at your changes too.

[Jason J. Herne (hernejj)]

> This sounds really exciting! You have followed the proper process for
> requesting such a merge but unfortunately I don't think we can get this into
> Lucid since we are past feature freeze. I'm requesting that the original
> author look at your changes too.

Hi Brian. Thanks for your interest. It's not problem for me if this has to wait for Maverick Meerkat.. I understand how the schedule works :). Once code review is complete I can always put an updated package in my PPA. Then others can test the changes if they are interested.

[lp:~hernejj/ubuntu-qa-tools/better_hugday]

339. By Jason J. Herne on 2010-04-13

- Added Google Chromium support

[Jason J. Herne (hernejj)]

One note I'd like to make for the reviewers.

The hugday code that I replaced was searching for two cookies: MOIN_SESSION and MOIN_ID. My new code only looks for MOIN_SESSION. I'm not sure what MOIN_ID is or where it comes from but I could not find any instances of it in my testing.

If there is a valid reason to continue to search for MOIN_ID as well as MOIN_SESSION then please let me know and I'll modify this branch to look for MOIN_ID if MOIN_SESSION is not found.

[Jason J. Herne (hernejj)]

Just a friendly poke to find out what the status is on this review. :)

[Markus Korn (thekorn)]

Hey Jason,
I'm sorry, I totally forgot about this merge proposal. Reviewing this huge diff will take some time, but I hope to get it done until next weekend.

Just a side note, have you ever thought about moving your cookielib into a separate project/package? Because it seems to be useful for other python scripts too, and as a side effect it would reduce the size of this diff.

Sorry again,
and thanks for the reminder,
have a nice day,
Markus

[Jason J. Herne (hernejj)]

On Mon, Aug 2, 2010 at 5:14 PM, Markus Korn <email address hidden> wrote:

> Hey Jason,
> I'm sorry, I totally forgot about this merge proposal. Reviewing this huge
> diff will take some time, but I hope to get it done until next weekend.
>
> Just a side note, have you ever thought about moving your cookielib into a
> separate project/package? Because it seems to be useful for other python
> scripts too, and as a side effect it would reduce the size of this diff.
>
> Sorry again,
> and thanks for the reminder,
> have a nice day,
> Markus
> --
>
> https://code.launchpad.net/~hernejj/ubuntu-qa-tools/better_hugday/+merge/23257<https://code.launchpad.net/%7Ehernejj/ubuntu-qa-tools/better_hugday/+merge/23257>
> You are the owner of lp:~hernejj/ubuntu-qa-tools/better_hugday.
>

I had considered separating out the cookielib code but I'm not sure what
other use it has at the moment. Seems preemptive to do it without another
project that can use it.
It can always be done as follow on work when another need arises. It would
not be that hard to do.

--
- Jason J. Herne (<email address hidden>)

Unmerged revisions

339. By Jason J. Herne on 2010-04-13

- Added Google Chromium support

338. By Jason J. Herne on 2010-04-08

- Fixed errors in usage information for hugday
- Added previously forgotten cookie loader file
- Updated readme and changelog

337. By Jason J. Herne on 2010-04-07

- ChromeCookieLoader.py: Fixed bug, need to check that cookies were loaded BEFORE we iterate through them ;)
- CookieFinder.py: Added missing import of os module
- CookieFinder.py: Added Konqueror support
- CookieFinder.py: Change browser string comparison to always use lower case
- CookieFinder.py: Fixed bug, In get_cookeies(), don't return if a cookie loader has no cookies, need to check ALL cookie loaders.
- KonquerorCookieLoader.py: Initial import.

336. By Jason J. Herne on 2010-04-07

- User can now specify exactly which browser to pull cookie file from: --browser=<firefox|seamonkey|chrome|epiphany>

335. By Jason J. Herne on 2010-04-07

- bug fix: Expand ~ in path names given by user
- Fixed error mesasge text
- Added print stmt to output the CookieLoader we're using

334. By jason <jason@foobarbaz> on 2010-04-07

- Fixed timestamp handling in Chrome

333. By jason <jason@foobarbaz> on 2010-04-06

- Removed unneeded comment from SeamonkeyLoader

332. By Jason J. Herne on 2010-04-05

- Properly close connection to sqlite database files

331. By Jason J. Herne on 2010-04-05

- Rewrote existing get_credentials code to use CookieFinder when the user specifies a cookie file on the command line.

330. By Jason J. Herne on 2010-04-05

- Bug fix: Check that path exists in cookier_scanner routine before scanning it.
- Bug fix: Uncomment exception handling code (originally commented for testing purposes.)
- Cookie Finder/Loader now provides a method to detect if cookie files were loaded.
- All cookie loaders have been updated to support the above method.
- hugday now uses the above method to give a proper error message if no cookie files were found.

=== modified file 'README'
--- README	2009-02-10 09:50:50 +0000
+++ README	2010-04-13 16:46:31 +0000
@@ -93,13 +93,26 @@
   $ hugday list --date 20090101 (for hugday at 2009-01-01)
   
 Before actually changing content of a wikipage the hugday tools needs
-to know the useres name and his wiki-id, 'hugday init' provides some
-options to get this information
-  $ hugday init --user <USER> --cookie <PATH> (try to get the MOIN_ID
-        out of the mozilla-like cookie file and use USER for entries in
-        the 'Triager' column)
+to know the user's Launchpad ID  and wiki-id, 'hugday init' provides some
+options to get this information:
+
+1) Let hugday automatically detect the cookie containing the wiki-id.
+
+  $ hugday init --user <USER> 
+
+2) Tell hugday which web brower you most recently used to log on to
+   wiki.ubuntu.com
+
+  $ hugday init --user <USER> --browser <firefox|seamonkey|chrome|epiphany|konqueror>
+
+3) Provide the path to a cookie file that contains a cookie for ".wiki.ubuntu.com"
+   that has the name "MOIN_SESSION".
+
+  $ hugday init --user <USER> --cookie <PATH> 
+
+4) Provide the wiki-id directly.
+
   $ hugday init --user <USER> --wiki-id <MOINID>
-  $ hugday init (interactive mode, not implemented yet)
   
 To mark a bug as DONE run:
   $ hugday close 123456

=== modified file 'debian/changelog'
--- debian/changelog	2010-03-31 19:17:37 +0000
+++ debian/changelog	2010-04-13 16:46:31 +0000
@@ -1,3 +1,14 @@
+ubuntu-qa-tools (0.1.4.4) lucid; urgency=low
+
+  * Improvements to cookie processing in hugday tool.
+   - Supported browsers: Firefox, Epiphany, Seamonkey, Google Chrome, Konqueror
+   - Automatic cookie detection for all supported browsers.
+   - user no longer has to close the browser to find cookies.
+   - Cookie file format automatically detected when --cookie is given
+   - User can specify a browser (--browser) and cookie is automatically found within that browser's cookie files.
+
+ -- Jason J. Herne <[e-mail]>  Thu, 8 Apr 2010 11:54:00 -0400
+
 ubuntu-qa-tools (0.1.4.3) lucid; urgency=low
 
   * Fix to work with launchpad's multi-version code.

=== modified file 'hugday-tools/hugday'
--- hugday-tools/hugday	2009-03-18 20:25:21 +0000
+++ hugday-tools/hugday	2010-04-13 16:46:31 +0000
@@ -19,13 +19,13 @@
 #
 # ##################################################################
 
-import sys
-import re
-import os
-import urllib
-import urllib2
-import libxml2
+import sys, os, shutil, time, re
+import urllib, urllib2, libxml2
 import cookielib
+import webbrowser
+
+from hugday_lib.cookie_lib.CookieFinder import CookieFinder
+
 try:
     import sqlite3 as sqlite
 except ImportError:
@@ -55,11 +55,13 @@
                   
 RE_BUGS = "%s(%%s)%s" %(re.escape("https://bugs.launchpad.net/bugs/"), re.escape("|"))
 
-
+######################################################
+# Parses Command Line using Python's optparse module #
+######################################################
 class CmdOptions(OptionParser):
     
     USAGE = (
-        "\t%prog init --user <USERNAME> [--wiki-id <MOINID>] [--cookie <PATH>]\n"
+        "\t%prog init --user <USERNAME> [--wiki-id <MOINID>] [--cookie <PATH>] [--browser <firefox|seamonkey|chrome|epiphany|konqueror>]\n"
         "\t%prog current [--remember]\n"
         "\t%prog close <Bugs> [--day <ID>] [--kde] [--user <TRIAGER>]\n"
         "\t%prog list [--day <ID>] [--kde] [--filter <open|done>]\n"
@@ -80,13 +82,15 @@
         make_option("--wiki-id", action="store", type="string", dest="wiki_id",
                     help="the users MOIN-ID"),
         make_option("--cookie", action="store", type="string", dest="cookie",
-                    help="path to users mozilla-like cookie file")
+                    help="path to users cookie file"),
+        make_option("--browser", action="store", type="string", dest="browser",
+                    help="Web browser that will be searched for the wiki-id cookie")
     )
             
     TOOLS = {
         "init": (
                  ("user", ),
-                 ("wiki_id", "cookie"),
+                 ("wiki_id", "cookie", "browser"),
                 ),
         "current": (
                     tuple(),
@@ -121,7 +125,7 @@
             else:
                 self.error("Only one sub-tool allowed")
         if tool == "close" and not options.bugs:
-            self.error("at elast one bug needed")
+            self.error("at least one bug needed")
         given_options = set(i for i, k in self.defaults.iteritems() if not getattr(options, i) == k)
         needed_options = set(CmdOptions.TOOLS[tool][0]) - given_options
         if needed_options:
@@ -250,44 +254,93 @@
         title = b.xpathEval("td[2]")[0].content.strip()
         triager = b.xpathEval("td[3]")[0].content.strip() or None
         yield (nr, title, done, triager, section)
-        
-def get_credentials(user, wiki_id=None, cookie=None):
+
+################################################################################
+# Obtain the user's moin_id. This will allow us to authenticate to the wiki in #
+# order to make updates.  The user may either provide the moin_id directly or  #
+# the user provides their browser cookie file from which we can read the ID.   #
+################################################################################
+def get_credentials(user, wiki_id=None, cookie_file_path=None, browser_str=None):
+
+    # User provided their moin_id string. Just save it in the config file.
     if wiki_id:
         config = update_config(user=user, moin_id=wiki_id)
-    elif cookie:
-        # parse cookie
-        # this includes a workaround for new cookie format
-        result = None
-        try:
-            cj = cookielib.MozillaCookieJar()
-            cj.load(os.path.expanduser(cookie))
-            result = [i.value for i in cj if i.domain == "wiki.ubuntu.com" \
-                and i.name in ("MOIN_ID", "MOIN_SESSION")]
-        except cookielib.LoadError:
-            if not sqlite:
-                raise TypeError(("Try to read cookiefile, cannot handle "
-                                 "format of '%s'" %cookie))
-            try:
-                con = sqlite.connect(cookie)
-                con.row_factory = sqlite.Row
-
-                cur = con.cursor()
-                cur.execute(("select value from moz_cookies where "
-                             "(name=? or name=?) and host=?"),
-                             ("MOIN_SESSION", "MOIN_ID", "wiki.ubuntu.com",))
-                #TODO: add check for expired cookie
-                result = [i["value"] for i in cur]
-            except sqlite.Error, e:
-                raise TypeError(("Error while trying to read cookie in "
-                                 "sql format, cannot handle "
-                                 "format of '%s'" %cookie))
-        if not result:
-            raise ValueError("No cookie with name 'MOIN_ID' found in '%s'" %cookie)
-        config = update_config(user=user, moin_id=result.pop())
+
+    # User provided their cookie file path. Attempt to read the MOIN_SESSION
+    # cookie from the cookie file.
+    elif cookie_file_path:
+
+        cookie_file_path = os.path.expanduser(cookie_file_path)
+
+        # Ensure cookie file exists
+        if (not os.path.exists(cookie_file_path)):
+            raise RuntimeError("The specified cookie file does not exist: '%s'" %cookie_file_path)
+
+        # Load cookie file
+        cookie_finder = CookieFinder()
+        cookie_file_valid = cookie_finder.set_cookie_file(cookie_file_path)
+
+        # Was cookie file valid?
+        if (not cookie_file_valid):
+            raise RuntimeError("The specified file is not a valid cookie file from a supported browser: '%s'.  " %cookie_file_path)
+
+        # Get desired cookie
+        cookie = cookie_finder.get_newest_cookie(domain=".wiki.ubuntu.com", name="MOIN_SESSION", ignore_expired=True)
+
+        # Did we find a matching cookie?
+        if (not cookie):
+            raise RuntimeError("Unable to find the MOIN_SESSION cookie in the given cookie file: '%s'. Please ensure you've logged in to wiki.ubuntu.com with this browser." %cookie_file_path )
+
+        # Found cookie! Update config file.
+        config = update_config(user=user, moin_id=cookie['value'])
+        print "Cookie detection succeeded."
+
+    # User provided a string telling us which browser they wish to
+    # search to find the wiki-id cookie.
+    elif browser_str:
+
+        # See if CookieFinder supports the user's browser
+        cookie_finder = CookieFinder()
+        browser_supported = cookie_finder.set_browser(browser_str)
+
+        if (not browser_supported):
+            raise RuntimeError("unsupported browser '%s'" % browser_str)
+
+        # Get desired cookie.
+        cookie = cookie_finder.get_newest_cookie(domain=".wiki.ubuntu.com", name="MOIN_SESSION", ignore_expired=True)
+
+        # Did the cookie loader find valid files to read cookies from?
+        if (not cookie_finder.has_cookie_files()):
+            raise RuntimeError("No valid cookie files were found for the given browser.")
+
+        # Did we find a matching cookie?
+        if (not cookie):
+            raise RuntimeError("Unable to find the MOIN_SESSION cookie in the given browser's cookie files: '%s'. Please ensure you've logged in to wiki.ubuntu.com with this browser." %browser_str)
+
+        # Found cookie! Update config file.
+        config = update_config(user=user, moin_id=cookie['value'])
+        print "Cookie detection succeeded."
+
+    # Only --user option was specified (--cookie & --wiki-id were omitted)
+    # Do auto detection of cookie file.
     else:
-        # interactive mode or openid, TBD
-        raise NotImplementedError
-        
+
+        # Try and automatically find the cookie we need
+        cookie_finder = CookieFinder()
+        cookie = cookie_finder.get_newest_cookie(domain=".wiki.ubuntu.com", name="MOIN_SESSION", ignore_expired=True)
+
+        # Did the cookie loader find valid files to read cookies from?
+        if (not cookie_finder.has_cookie_files()):
+            raise RuntimeError("No valid cookie files were found. You do not seem to be using any supported browsers.")
+
+        # Did we find a matching cookie?
+        if (not cookie):
+            raise RuntimeError("Unable to detect the MOIN_SESSION cookie. Please ensure you've logged in to wiki.ubuntu.com with a supported browser.")
+       
+        # Write the obtained moin_id to the user's configuration file
+        print "Cookie detection succeeded."
+        config = update_config(user=user, moin_id=cookie['value'])
+
 def build_request(url, moin_id, proxy=None, data=None):
     request = urllib2.Request(url, data)
     #~ # moinmoin < 1.6
@@ -379,18 +432,27 @@
         url = "%s/KDE" %url
     return url
 
+
+##########################
+# Main Program           #
+# Execution starts here  #
+##########################
 def main():
+
     cmdoptions = CmdOptions()
     tool, options = cmdoptions.parse_args()
+
     if tool == "list-days":
         print "list of all hugdays, latest first:"
         for day in parse_all_hugdays():
             print "\t* https://wiki.ubuntu.com/UbuntuBugDay/%s" %day
+
     elif tool == "current":
         current = get_current()
         print current
         if options.remember:
             update_config(current=current)
+
     elif tool == "list":
         url = get_url(options.day, options.kde)
         i = None
@@ -399,15 +461,17 @@
         if i is not None:
             print "="*50
             print "Total:", i + 1
+
     elif tool == "init":
-        get_credentials(options.user, options.wiki_id, options.cookie)
+        get_credentials(options.user, options.wiki_id, options.cookie, options.browser)
+
     elif tool == "close":
         url = get_url(options.day, options.kde)
         close_bugs(options.bugs, url, options.user)
+
+    # FIXME: Shouldn't this be impossible since we detect this in cmdoptions.parse_args()?
     else:
         raise RuntimeError("unknown tool '%s'" %tool)
-        
-        
 
 if __name__ == "__main__":
     try:

=== added directory 'hugday_lib'
=== added file 'hugday_lib/__init__.py'
=== added directory 'hugday_lib/cookie_lib'
=== added file 'hugday_lib/cookie_lib/ChromeCookieLoader.py'
--- hugday_lib/cookie_lib/ChromeCookieLoader.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/ChromeCookieLoader.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,63 @@
+#!/usr/bin/python
+
+import os, shutil
+
+from CookieLoader import CookieLoader
+
+# This is a subclass of CookieLoader.  Please see CookieLoader for more details.
+class ChromeCookieLoader(CookieLoader):
+
+    # browser name
+    browser_str = "chrome"
+
+    # List of directories to search for cookie files.
+    cookie_file_search_path = [ os.path.join( "~" , ".config" , "google-chrome" ) ,
+                                os.path.join( "~" , ".config" , "chromium" )
+                              ]
+    
+    # List of valid cookie file names.
+    cookie_file_names = [ "Cookies" ]
+
+    # attempts to load cookies from the given file.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed then this function will return True. If it returns false then
+    # the specified cookie file is not readable by this cookie finder.
+    def load_cookies_from_file(self, cookie_file_path):
+
+        # If the given cookie file does not exist, give up.
+        if (not os.path.exists(cookie_file_path)):
+            return False
+
+        # If we attempt to use the cookie file while the browser is running
+        # we will get errors. Only one process is allowed to use it at a time.
+        # To get around this we can simply copy it to /tmp and then use it.
+        tmp_cookie_file_path = os.path.join( "/" , "tmp" , os.path.basename(cookie_file_path) )
+        shutil.copyfile(cookie_file_path, tmp_cookie_file_path)
+
+        # Chrome cookie files are sqlite databases
+        cookies = self.load_sqlite_cookie_file(tmp_cookie_file_path, table_name="cookies", domain_key="host_key", name_key="name", value_key="value", expire_time_key="expires_utc")
+
+        # If the load failed, then we've been given an invalid cookie file.
+        if (not cookies): 
+            os.remove(tmp_cookie_file_path)
+            return False
+
+        # Fix timestamps in Google Chrome cookies!!
+        # Chrome stores cookies with a 64-bit integer that represents the number of Microseconds that have passed since 1/1/1601. So it's not quite a
+        # Unix timestamp.  See top comments in this file for more info: http://src.chromium.org/svn/trunk/src/base/time_posix.cc
+        #
+        # The basic proceedure for converting a Chrome timestamp into a Unix timestap is this: unix_time = (chrome_time / 1000000) - 11644473600
+        for cookie in cookies:
+            cookie['expire_time'] = (cookie['expire_time'] / 1000000) - 11644473600
+
+        # Loop over all cookies and add them to our cookies list
+        for cookie in cookies:
+            self.cookies.append(cookie)
+
+        # Erase temporary cookie file.
+        os.remove(tmp_cookie_file_path)
+
+        # We were able to read cookies from this file.
+        self.found_valid_cookie_file = True
+        return True
+

=== added file 'hugday_lib/cookie_lib/CookieFinder.py'
--- hugday_lib/cookie_lib/CookieFinder.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/CookieFinder.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,149 @@
+from FirefoxCookieLoader import FirefoxCookieLoader
+from EpiphanyCookieLoader import EpiphanyCookieLoader
+from SeamonkeyCookieLoader import SeamonkeyCookieLoader
+from ChromeCookieLoader import ChromeCookieLoader
+from KonquerorCookieLoader import KonquerorCookieLoader
+
+import time, os
+from datetime import datetime
+
+# Used to find/access cookies for many different web browsers.  You can point
+# CookieFinder to a specific cookie file (set_cookie_file) or you can simply
+# perform a search (get_cookies/get_newest_cookie) and CookieFinder will 
+# automatically scan every cookie file it can find for every browser it supports.
+class CookieFinder:
+
+    # List of cookie loaders. Each one scans a different browser's cookie file(s).
+    cookie_loaders = []
+
+    # Creates a new CookieFinder.
+    def __init__(self):
+
+        # Populate cookie loaders, one loader for each browser we support
+        self.cookie_loaders.append( FirefoxCookieLoader() )
+        self.cookie_loaders.append( EpiphanyCookieLoader() )
+        self.cookie_loaders.append( SeamonkeyCookieLoader() )
+        self.cookie_loaders.append( ChromeCookieLoader() )
+        self.cookie_loaders.append( KonquerorCookieLoader() )
+
+    # Tells CookieFinder that you only wish to search the given browser for
+    # cookies.  All CookieLoaders will be search until one is found that matches
+    # the given browser string. Once that loader is found all others will be
+    # removed from our loaders list. All subsequent operations using this CookieFiner
+    # will only invoke the one loader.  
+    # Returns True if a suitable CookieLoader was found. Returns False otherwise.
+    def set_browser(self, browser_str):
+
+        # Avoid case comparison problems
+        browser_str = browser_str.lower()
+
+        # Locate a cookie loader for the browser the user specified
+        for cookie_loader in self.cookie_loaders:
+
+            if ( cookie_loader.browser_str == browser_str):
+
+                # We found a cookie loader that supports the user's browser.
+                # Throw away the rest of the loaders.
+                self.cookie_loaders = [cookie_loader]
+
+                # Return true and the caller will be most pleased :)
+                return True
+
+        # We got to the end of the list and none of our cookie loaders match.
+        return False
+
+    # Tells CookieFinder that you only want to search the given file for cookies.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed by CookieFinder then this function will return True. If it returns
+    # false then the specified cookie file is not readable by CookieFinder.
+    def set_cookie_file(self, cookie_file_path):
+
+        # Ensure ~ is properly handled in the path
+        cookie_file_path = os.path.expanduser(cookie_file_path)
+
+        # Locate a cookie loader that can read this cookie file.
+        for cookie_loader in self.cookie_loaders:
+
+            if ( cookie_loader.load_cookies_from_file(cookie_file_path) ):
+
+                # Since we found a cookie loader that can read this file there is
+                # no need to keep the rest around.
+                print "Using Cookie Loader " + str(cookie_loader.__module__)
+                self.cookie_loaders = [cookie_loader]
+
+                # We found a cookie loader that will read the caller's cookie
+                # file.  Return true and the caller will be most pleased :)
+                return True
+
+        # We got to the end of the list and none of our cookie loaders read the file.
+        return False
+
+    # Returns a list of cookies found by this CookieFinder that match the given criteria.
+    # If caller does not specify any of the search criteria then all cookies will be
+    # returned. An empty list is returned if no matching cookies were found.
+    def get_cookies(self, domain=None, name=None, ignore_expired=False):
+
+        cookies = []
+
+        for cookie_loader in self.cookie_loaders:
+
+            # Ensure this cookie loader has its cookies loaded.
+            found_cookies = cookie_loader.load_cookies()
+
+            # If no cookies were found go try next loader
+            if(not found_cookies): continue
+
+            # Look for cookies matching the caller's search criteria.
+            for cookie in cookie_loader.cookies:
+                # If user specified a domain, skip this cookie if its domain does not match
+                # The domain name is converted to lowercase to avoid comparison failue.
+                # Any leading '.' character is also removed from domain name to prevent comparison failure.
+                if domain and (not cookie['domain'].lower().lstrip(".") == domain.lower().lstrip(".")):
+                    continue
+
+                # If user specified a name, skip this cookie if its name does not match
+                if name and (not cookie['name'].lower() == name.lower()):
+                    continue
+
+                # If user wants: skip this cookie if it has expired.
+                if ignore_expired and (time.time() > cookie['expire_time']):
+                    continue
+
+                # This cookie matches, add it to the list.
+                #print ( "  Cookie: name=%s   value=%s   expires=%s" % (cookie['name'], cookie['value'], datetime.fromtimestamp(cookie['expire_time'])) )
+                cookies.append(cookie)
+
+        # Return cookie list
+        return cookies
+
+    # Returns the newest cookie found by this CookieFinder that match the given criteria.
+    # The "newest" cookie is the cookie with the latest expiration date. This is really
+    # only meaningful when comparing the same cookie but from different browser files.
+    # It is useful when you just want ONE copy (the latest and greatest) of a specific
+    # cookie but you're not sure which browser or cookie file has it.
+    # None is returned if no matching cookie is found.
+    def get_newest_cookie(self, domain=None, name=None, ignore_expired=False):
+
+        # Get list of all matching cookies
+        cookies = self.get_cookies(domain, name, ignore_expired)
+        if (len(cookies) == 0): return None
+
+        newest_cookie = cookies[0]
+
+        # Look for the one with the largest expire time.
+        for cookie in cookies:
+            if ( cookie['expire_time'] > newest_cookie['expire_time'] ):
+                newest_cookie = cookie
+
+        return newest_cookie
+
+    # Returns True if this CookieFinder was able to find/load at least one cookie file.
+    # This is useful to distinguish between thw following cases: 1) No cookie files are
+    # found to load cookies from.  2) No matching cookies were found inside the cookie file(s).
+    def has_cookie_files(self):
+
+        for cookie_loader in self.cookie_loaders:
+            if (cookie_loader.found_valid_cookie_file): return True
+
+        return False
+

=== added file 'hugday_lib/cookie_lib/CookieLoader.py'
--- hugday_lib/cookie_lib/CookieLoader.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/CookieLoader.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,209 @@
+#!/usr/bin/python
+
+import os, cookielib
+
+try: import sqlite3 as sqlite
+except ImportError:
+    try: from pysqlite2 import dbapi2 as sqlite
+    except ImportError: sqlite = None
+        
+# base CookieLoader class. This class is not mean to be used directly. Instead
+# child classes should be created to implement cookie loading for a specific
+# browser.  
+#  
+# Used to find/access cookies for some web browser.  You can specify
+# a specific cookie file (load_cookies_from_file) or you can let this class
+# automatically detect cookie files and load cookies from them (load_cookies).
+class CookieLoader:
+
+    # Child classes should provide this string.
+    # all lower case String (no spaces) that represents the name of the webbrowser that
+    # this CookieLoader loads cookies for.
+    browser_str = None
+
+    # List of cookies found. This is not valid until either load_cookies_from_file
+    # has been called and has returned True or load_cookies has been called and
+    # has returned true.
+    #
+    # cookie objects on this list are dictionaries and have the following properties:
+    # 'domain'        : string - Host setting the cookie
+    # 'name'         : string - Cookie name
+    # 'value'        : string - Cookie value
+    # 'expire_time'  : integer - Expire date/time (Unix Time)
+    cookies = []
+
+    # Indicates that at least one valid cookie file was found during loading. 
+    found_valid_cookie_file = False
+
+    # Child classes should provide this list.
+    # List of directories to search for cookie files (searched recursively).
+    cookie_file_search_path = []
+
+    # Child classes should provide this list.
+    # List of valid cookie file names. Files with these names will be search for cookies
+    # if they are found in the paths listed in the cookie_file_search_path.
+    cookie_file_names = []
+
+    # Child classes should provide this method.
+    # attempts to load cookies from the given file.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed then this function will return True. If it returns false then
+    # the specified cookie file is not readable by this cookie finder.
+    def load_cookies_from_file(self, cookie_file_path):
+       raise NotImplementedError("This method cannot be used directly. Subclass CookieFinder and override this method.")
+
+    # Loads all cookies that are found in all cookie files found.
+    # Cookie files are searched for in the paths found in 
+    # the cookie_file_search_paths list.
+    # Returns True if at least one cookie file can be loaded, False otherwise.
+    def load_cookies(self):
+
+        # If there are already cookies loaded, do not reload them, just return.
+        # This prevents reloading the cookie list on every search since load_cookies 
+        # is called before every search.
+        if (self.cookies):
+            return ( len(self.cookies) > 0 )
+
+        # Initialize cookies to an empty list
+        self.cookies = []
+        
+        found_cookies = False
+
+        # Find all cookie files
+        cookie_file_paths = self.find_cookie_files()
+
+        # Try to load each cookie file individually. If we can load any one of them then
+        # we can declare succcess.
+        for cookie_file_path in cookie_file_paths:
+
+            # If cookies are found, they are added to this objects cookies list.
+            found_cookies = self.load_cookies_from_file(cookie_file_path)
+
+        return found_cookies
+
+    ######################################################
+    ################# Internal Functions #################
+    ######################################################
+
+    # Searches all predefined paths for cookie files. Returns a list of fully
+    # qualified paths to cookie files found.
+    def find_cookie_files(self):
+
+        cookie_file_paths = []
+
+        # Search each directory for cookies
+        for cur_dir in self.cookie_file_search_path:
+
+            # Expand '~' in paths
+            cur_dir = os.path.expanduser(cur_dir)
+
+            cur_paths = self.cookie_file_scan(cur_dir, cookie_file_paths)
+
+        return cookie_file_paths
+
+    # Recursively scan for cookie files in the target directory and return a
+    # list of the cookie files found.
+    def cookie_file_scan(self, target_dir, cookie_file_list):
+
+        # Ensure current directory exists
+        if (not os.path.exists(target_dir)): return
+
+        # Loop over all files/dirs in the current_dir
+        for cur_file in os.listdir(target_dir):
+
+            # Full path to current file
+            cur_file_path = os.path.join(target_dir, cur_file)
+
+            # If the current file is a folder, recursively process it
+            if( os.path.isdir(cur_file_path)):
+
+                # Scan it for cookie files
+                self.cookie_file_scan(cur_file_path, cookie_file_list)
+
+            # The current file is a file (not a directory), see if it is a cookie file.
+            else:
+
+                # If the current file matches any of the names in cookie_file_names, it is a cookie file.
+                for name in self.cookie_file_names:
+                    if (cur_file.lower() == name.lower()):
+                        cookie_file_list.append(cur_file_path)
+
+    ####################################################################
+    ################# Generic Cookie Parsing Functions #################
+    ####################################################################
+    # These parsing functions can be used by many different browsers.  #
+    # They are provided here for child classes to use. This avoid      #
+    # duplicating the code in each child class that can use them.      #
+    ####################################################################
+
+    # Loads cookies from a sqlite cookie file and returns the list of cookies.
+    # Returns list of cookies on success and None otherwise.
+    # The table and key names default to the ones used by Mozilla based browsers as they are the most common.
+    def load_sqlite_cookie_file(self, cookie_file_path, table_name="moz_cookies", domain_key="host", name_key="name", value_key="value", expire_time_key="expiry"):
+
+        cookies = []
+
+        # "Connect" to cookie file :)
+        try:
+            con = sqlite.connect(cookie_file_path)
+            con.row_factory = sqlite.Row
+
+            # Perform SQL query to obtain all cookies
+            cur = con.cursor().execute("select * from " + table_name)
+
+        # If query fails, this is NOT a valid sqlite cookie file.
+        except sqlite.Error, e:
+            return None
+
+        # cookie objects returned are dictionaries and have the following properties
+        # 'id'          - ???
+        # 'name'        - Cookie name
+        #'value'        - Cookie value
+        #'host'         - Host setting the cookie
+        #'path'         - ???
+        #'expiry'       - Expire date/time (Unix Time)
+        #'lastAccessed' - last time cookie was used.
+        #'isSecure'     - ???
+        #'isHttpOnly'   - ???
+
+
+        # Build and return list of cookies.
+        for cookie in cur:
+            new_cookie = {}
+            new_cookie['domain'] = cookie[domain_key]
+            new_cookie['name'] = cookie[name_key]
+            new_cookie['value'] = cookie[value_key]
+            new_cookie['expire_time'] = cookie[expire_time_key]
+            cookies.append(new_cookie)
+
+        # Close connection to database file
+        con.close()
+
+        return cookies
+
+    # Loads cookies from a plain text cookie file and returns the list of cookies.
+    # Returns list of cookies on success and None otherwise.
+    def load_txt_cookie_file(self, cookie_file_path):
+
+        cookies = []
+
+        # Load cookie file
+        try:
+            cookie_jar = cookielib.MozillaCookieJar()
+            cookie_jar.load(cookie_file_path)
+
+        # If load fails, this is NOT a valid plain text cookie file.
+        except cookielib.LoadError:
+            return None
+
+        # Loop over all cookies and all them to our cookies list
+        for cookie in cookie_jar:
+            new_cookie = {}
+            new_cookie['domain'] = cookie.domain
+            new_cookie['name'] = cookie.name
+            new_cookie['value'] = cookie.value
+            new_cookie['expire_time'] = cookie.expires # int
+            cookies.append(new_cookie)
+ 
+        return cookies
+

=== added file 'hugday_lib/cookie_lib/EpiphanyCookieLoader.py'
--- hugday_lib/cookie_lib/EpiphanyCookieLoader.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/EpiphanyCookieLoader.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,53 @@
+#!/usr/bin/python
+
+import os, shutil
+
+from CookieLoader import CookieLoader
+
+# This is a subclass of CookieLoader.  Please see CookieLoader for more details.
+class EpiphanyCookieLoader(CookieLoader):
+
+    # browser name
+    browser_str = "epiphany"
+
+    # List of directories to search for cookie files.
+    cookie_file_search_path = [ os.path.join( "~" , ".gnome2" , "epiphany" ) ]
+
+    # List of valid cookie file names.
+    cookie_file_names = [ "cookies.sqlite" ]
+
+    # attempts to load cookies from the given file.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed then this function will return True. If it returns false then
+    # the specified cookie file is not readable by this cookie finder.
+    def load_cookies_from_file(self, cookie_file_path):
+
+        # If the given cookie file does not exist, give up.
+        if (not os.path.exists(cookie_file_path)):
+            return False
+
+        # If we attempt to use the cookie file while the browser is running
+        # we will get errors. Only one process is allowed to use it at a time.
+        # To get around this we can simply copy it to /tmp and then use it.
+        tmp_cookie_file_path = os.path.join( "/" , "tmp" , os.path.basename(cookie_file_path) )
+        shutil.copyfile(cookie_file_path, tmp_cookie_file_path)
+
+        # Epiphany cookie files are sqlite databases
+        cookies = self.load_sqlite_cookie_file(tmp_cookie_file_path)
+
+        # If the load failed, then we've been given an invalid cookie file.
+        if (not cookies): 
+            os.remove(tmp_cookie_file_path)
+            return False
+
+        # Loop over all cookies and add them to our cookies list
+        for cookie in cookies:
+            self.cookies.append(cookie)
+
+        # Erase temporary cookie file.
+        os.remove(tmp_cookie_file_path)
+
+        # We were able to read cookies from this file.
+        self.found_valid_cookie_file = True
+        return True
+

=== added file 'hugday_lib/cookie_lib/FirefoxCookieLoader.py'
--- hugday_lib/cookie_lib/FirefoxCookieLoader.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/FirefoxCookieLoader.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,60 @@
+#!/usr/bin/python
+
+import os, shutil
+
+from CookieLoader import CookieLoader
+
+# This is a subclass of CookieLoader.  Please see CookieLoader for more details.
+class FirefoxCookieLoader(CookieLoader):
+
+    # browser name
+    browser_str = "firefox"
+    
+    # List of directories to search for cookie files.
+    cookie_file_search_path = [ os.path.join( "~" , ".mozilla" , "firefox" ) ]
+
+    # List of valid cookie file names.
+    cookie_file_names = [ "cookies.sqlite" , "cookies.txt" ]
+
+    # attempts to load cookies from the given file.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed then this function will return True. If it returns false then
+    # the specified cookie file is not readable by this cookie finder.
+    def load_cookies_from_file(self, cookie_file_path):
+
+        # If the given cookie file does not exist, give up.
+        if (not os.path.exists(cookie_file_path)):
+            return False
+
+        # If we attempt to use the cookie file while the browser is running
+        # we will get errors. Only one process is allowed to use it at a time.
+        # To get around this we can simply copy it to /tmp and then use it.
+        tmp_cookie_file_path = os.path.join( "/" , "tmp" , os.path.basename(cookie_file_path) )
+        shutil.copyfile(cookie_file_path, tmp_cookie_file_path)
+
+        # Firefox cookie files can be one of two formats:
+        # old format - cookies.txt    - Plain text cookie file 
+        # new format - cookies.sqlite - sqlite database file
+
+        # 1st, try to read that file as if it is an sqlite cookie file.
+        cookies = self.load_sqlite_cookie_file(tmp_cookie_file_path)
+
+        # If that fails, try to read it as plain text
+        if (not cookies): cookies = self.load_txt_cookie_file(tmp_cookie_file_path)
+
+        # If they both failed, then we've been given an invalid cookie file.
+        if (not cookies): 
+            os.remove(tmp_cookie_file_path)
+            return False
+
+        # Loop over all cookies and add them to our cookies list
+        for cookie in cookies:
+            self.cookies.append(cookie)
+
+        # Erase temporary cookie file.
+        os.remove(tmp_cookie_file_path)
+
+        # We were able to read cookies from this file.
+        self.found_valid_cookie_file = True
+        return True
+

=== added file 'hugday_lib/cookie_lib/KonquerorCookieLoader.py'
--- hugday_lib/cookie_lib/KonquerorCookieLoader.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/KonquerorCookieLoader.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,111 @@
+ #!/usr/bin/python
+
+import os, shutil
+
+from CookieLoader import CookieLoader
+
+# This is a subclass of CookieLoader.  Please see CookieLoader for more details.
+class  KonquerorCookieLoader(CookieLoader):
+
+    # browser name
+    browser_str = "konqueror"
+    
+    # List of directories to search for cookie files.
+    cookie_file_search_path = [ os.path.join( "~" , ".kde" , "share" , "apps" , "kcookiejar") ]
+
+    # List of valid cookie file names.
+    cookie_file_names = [ "cookies" ]
+
+    # attempts to load cookies from the given file.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed then this function will return True. If it returns false then
+    # the specified cookie file is not readable by this cookie finder.
+    def load_cookies_from_file(self, cookie_file_path):
+
+        # If the given cookie file does not exist, give up.
+        if (not os.path.exists(cookie_file_path)):
+            return False
+
+        # Konqueror cookie files are in plain text format
+        cookies = self.load_konqueror_cookie_file(cookie_file_path)
+
+        # If the load failed, then we've been given an invalid cookie file.
+        if (not cookies): 
+            print "not cookies"
+            return False
+
+        # Loop over all cookies and add them to our cookies list
+        for cookie in cookies:
+            self.cookies.append(cookie)
+
+        # We were able to read cookies from this file.
+        return True
+
+
+
+    # Loads cookies from a Konqueror plain text cookie file and returns the list of cookies.
+    # Returns list of cookies on success and None otherwise.
+    def load_konqueror_cookie_file(self, cookie_file_path):
+
+        cookies = []
+
+        # Here is an example Konquerir cookie file:
+        # # KDE Cookie File v2
+        # #
+        # # Host               Domain               Path         Exp.date   Prot Name                 Sec  Value
+        # [.ubuntu.com]
+        # wiki.ubuntu.com      ".wiki.ubuntu.com"   "/"          1286259931    0 __utmz               4    230736537.1270491931.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none)
+        # wiki.ubuntu.com      ".wiki.ubuntu.com"   "/"          1333564241    0 __utma               4    230736537.1618083871.1270491931.1270492211.1270492242.3
+        # wiki.ubuntu.com      ".wiki.ubuntu.com"   "/"          1270494047    0 __utmb               4    230736537
+        # wiki.ubuntu.com      ".wiki.ubuntu.com"   "/"          1333564247    0 __utmv               4    230736537.UbuntuWiki
+        # wiki.ubuntu.com      ""                   "/"          1585852247    0 MOIN_SESSION         4    249_4_7z1eln24s28t8ifido5k3cnwc_
+        # [.launchpad.net]
+        # login.launchpad.net  ""                   "/"          1271701555    0 sessionid            5    9ebf071969502599c1d1f959d210c892
+        # [.slashdot.org]
+        # slashdot.org         ".slashdot.org"      "/"          1286260261    0 __utmz               4    9273847.1270492261.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
+        # slashdot.org         ".slashdot.org"      "/"          1333564271    0 __utma               4    9273847.870340038.1270492261.1270492261.1270492261.1
+        # slashdot.org         ".slashdot.org"      "/"          1270494071    0 __utmb               4    9273847.1.10.1270492261
+        #
+
+        # Open cookie file
+        cookie_file = open(cookie_file_path, "r")
+
+        # Ensure this is a Konqueror cookie file
+        first_line = cookie_file.readline()
+        if ( first_line.find("KDE Cookie File") == -1 ):
+            print "Not cookie file!"
+            return False
+        
+        # Parse file line by line looking for cookies...
+        for line in cookie_file:
+
+            # Throw away comments, host headers
+            if ( line.startswith("#") ): continue
+            if ( line.startswith("[") ): continue
+
+            # Split string on whitespace (Ignore lines that do not contain enough tokens)
+            tokens = line.split()
+            if ( len(tokens) < 8): continue
+
+            # Construct cookie
+            new_cookie = {}
+            new_cookie['domain'] = tokens[0]
+            new_cookie['name'] = tokens[5]
+            new_cookie['value'] = tokens[7]
+            new_cookie['expire_time'] = tokens[3]
+
+            # Sanity check data & convert expire time to an integer
+            if (new_cookie['expire_time'].isdigit()):
+                new_cookie['expire_time'] = int(new_cookie['expire_time'])
+            else: continue
+
+            # We have a good cookie :)
+            cookies.append(new_cookie)
+
+        # Close file.
+        cookie_file.close()
+
+        # Found cookie files :)
+        self.found_valid_cookie_file = True
+        return cookies
+

=== added file 'hugday_lib/cookie_lib/SeamonkeyCookieLoader.py'
--- hugday_lib/cookie_lib/SeamonkeyCookieLoader.py	1970-01-01 00:00:00 +0000
+++ hugday_lib/cookie_lib/SeamonkeyCookieLoader.py	2010-04-13 16:46:31 +0000
@@ -0,0 +1,43 @@
+#!/usr/bin/python
+
+import os, shutil
+
+from CookieLoader import CookieLoader
+
+# This is a subclass of CookieLoader.  Please see CookieLoader for more details.
+class SeamonkeyCookieLoader(CookieLoader):
+
+    # browser name
+    browser_str = "seamonkey"
+
+    # List of directories to search for cookie files.
+    cookie_file_search_path = [ os.path.join( "~" , ".mozilla" , "default" ) ]
+
+    # List of valid cookie file names.
+    cookie_file_names = [ "cookies.txt" ]
+
+    # attempts to load cookies from the given file.
+    # If cookie_file_path points to a valid readable cookie file that can be 
+    # parsed then this function will return True. If it returns false then
+    # the specified cookie file is not readable by this cookie finder.
+    def load_cookies_from_file(self, cookie_file_path):
+
+        # If the given cookie file does not exist, give up.
+        if (not os.path.exists(cookie_file_path)):
+            return False
+
+        # Epiphany cookie files are sqlite databases
+        cookies = self.load_txt_cookie_file(cookie_file_path)
+
+        # If the load failed, then we've been given an invalid cookie file.
+        if (not cookies): 
+            return False
+
+        # Loop over all cookies and add them to our cookies list
+        for cookie in cookies:
+            self.cookies.append(cookie)
+
+        # We were able to read cookies from this file.
+        self.found_valid_cookie_file = True
+        return True
+

=== added file 'hugday_lib/cookie_lib/__init__.py'
=== modified file 'setup.py'
--- setup.py	2010-03-31 16:37:16 +0000
+++ setup.py	2010-04-13 16:46:31 +0000
@@ -28,5 +28,7 @@
                'dl-ubuntu-test-iso/iso-ripper',
                'hugday-tools/hugday',
             ],
-    packages=[],
+    packages=['hugday_lib',
+              'hugday_lib/cookie_lib'
+            ]
 )