Merge lp:~sergiusens/phablet-tools/flash_change into lp:phablet-tools

[Sergio Schvezov]

Refactoring phablet-flash to accommodate multiple flashing scenarios. Adding community flashing support.

Don't worry about the conflicts for now. Just branch straight and check the code if you are reviewing code.

Major change:
positional params: community, cdimage-touch, cdimage-legacy, ubuntu-system

To flash and ubuntu image based upgrade:
phablet-flash ubuntu-system

To flash cdimage touch
phablet-flash cdimage-touch (--pending to get pending)

cdimage legacy:
phablet-flash cdimage-legacy (--list-revisions, --latest and --revision work as usual, just not exposed to other params)

community:
phablet-flash community --device i9100 (will download, not sure if flash as I don't have the device).

I'm not using /cache/recovery/command anymore, instead /cache/recovery/extendedcomands to support community deploys.

Feel free to test and report early issues. I'm still shuffling some code around. You might notice some duplication, that's on purpose, things that might look similar today may change tomorrow and I've had a hard time with this already.

Still missing: adhoc positional argument and device and ubuntu override (I guess img too, but we aren't distributing those).

[Daniel Holbach (dholbach)]

daniel@daydream:~/flash_change$ ./phablet-flash community --device i9100
Traceback (most recent call last):
  File "./phablet-flash", line 24, in <module>
    from phabletutils import arguments
  File "/home/daniel/flash_change/phabletutils/arguments.py", line 21, in <module>
    from phabletutils import environment
  File "/home/daniel/flash_change/phabletutils/environment.py", line 26, in <module>
    from phabletutils import community
ImportError: cannot import name community
daniel@daydream:~/flash_change$

Could it be you forgot to "bzr add community*" somewhere?

[Daniel Holbach (dholbach)]

A few small things:
phablet-flash:17: 'argparse' imported but unused
phablet-flash:20: 'subprocess' imported but unused
phablet-flash:22: 'sleep' imported but unused
phablet-flash:25: 'environment' imported but unused
phablet-flash:27: 'ubuntuimage' imported but unused

Apart from that the code looks cleaner now. There were some things I was a bit unsure about, like when adb is run, and with which arguments, but it seems like a lot of the code was just moved somewhere else.

Good work and thanks for fixing bug 1201811.

review: Approve

[Sergio Schvezov (sergiusens)]

> A few small things:
> phablet-flash:17: 'argparse' imported but unused
> phablet-flash:20: 'subprocess' imported but unused
> phablet-flash:22: 'sleep' imported but unused
> phablet-flash:25: 'environment' imported but unused
> phablet-flash:27: 'ubuntuimage' imported but unused

fixed in 169

> Apart from that the code looks cleaner now. There were some things I was a bit
> unsure about, like when adb is run, and with which arguments, but it seems
> like a lot of the code was just moved somewhere else.

thanks, it's easy when you can break command line backwards compatibility :-)

> Good work and thanks for fixing bug 1201811.

Was fun :-D

[Andy Doan (doanac)]

looks like the flock logic from http://bazaar.launchpad.net/~phablet-team/phablet-tools/trunk/revision/148 is being lost?

[Sergio Schvezov (sergiusens)]

> looks like the flock logic from http://bazaar.launchpad.net/~phablet-team
> /phablet-tools/trunk/revision/148 is being lost?

oh, I didn't apply it yet, but it will be much easier here :-)

[Daniel Holbach (dholbach)]

Does anyone else want to comment?

[Sergio Schvezov (sergiusens)]

> looks like the flock logic from http://bazaar.launchpad.net/~phablet-team
> /phablet-tools/trunk/revision/148 is being lost?

It's here now
http://bazaar.launchpad.net/~sergiusens/phablet-tools/flash_change/revision/176

[Andy Doan (doanac)]

revno 176 adds the flock back. due to the way your code is now structured you could probably rename the the "flocked" function to be "_flocked" since its only used in that module. not a big deal though.

I tested the code though and it still works for me. We may want to send an email out just before this lands in the archive so that people are aware. For instance, its going to require us to update our provision script in automation.

review: Approve

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:179
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~sergiusens/phablet-tools/flash_change/+merge/177927/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/125/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/68
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/68
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/68

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/125/rebuild

review: Needs Fixing (continuous-integration)

[Daniel Holbach (dholbach)]

A small thing I just noticed: please replace "bazaar" with "bzr" in debian/control.

review: Needs Fixing

[Sergio Schvezov (sergiusens)]

> A small thing I just noticed: please replace "bazaar" with "bzr" in
> debian/control.

revno 180 has this fix

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:182
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/126/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/69
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/69
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/69

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/126/rebuild

review: Approve (continuous-integration)

[Daniel Holbach (dholbach)]

Good work!

review: Approve

[Paul Larson (pwlars)]

1186 + log.info('Download directory set to %s' % download_dir)
1187 + if not os.path.exists(download_dir):
1188 + log.info('Creating %s' % download_dir)
1189 + os.makedirs(download_dir)
This is still going to suffer from the makedirs race described in https://bugs.launchpad.net/phablet-tools/+bug/1209408

Otherwise, looks good.

review: Approve

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:184
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/127/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/70
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/70
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/70

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/127/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:185
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/128/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/71
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/71
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/71

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/128/rebuild

review: Approve (continuous-integration)

[Ricardo Salveti (rsalveti)]

A few comments:
- Can you make --help to display all possible options by default? 'phablet-flash cdimage-touch -h' works, but 'phablet-flash -h cdimage-touch' doesn't. Also see that when you get a help from a specific group, the default ones are not part of the list (like -s and -D).

- Would it be possible to add a man page for phablet-flash as well?

- Please add a way for it to display the license when flashing a community based port. If possible I'd make this a requirement for a community build, as tag the stamp differently per device/port, so people know what they are flashing.

- Can you also remove the generated tmp files after flashing the device? Check:
INFO:phablet-flash:Pushing /tmp/tmpPNd_Gv to /cache/recovery/extendedcommand
....
rsalveti@evatp:~$ cat /tmp/tmpPNd_Gv
mount("/sdcard/");
install_zip("/sdcard/saucy-preinstalled-touch-armel+manta.zip");
install_zip("/sdcard/saucy-preinstalled-touch-armhf.zip");

Code:
721 + parser.add_argument('-d', '--device', required=True,
722 + help='Specify device to flash')

Mind adding a better help statement here? You could also link at a wiki or somewhere explaining how people can look for the supported devices. Later on we could also have a --list option to show all the possible devices to flash.

review: Needs Fixing

[Ricardo Salveti (rsalveti)]

Other than that, it works as expected. Flashed all the different images (for community I just downloaded it as I don't have the device), and didn't find any issue.

Would just need a major announcement as the script arguments are quite different from the previous one.

[Sergio Schvezov (sergiusens)]

> A few comments:
> - Can you make --help to display all possible options by default? 'phablet-
> flash cdimage-touch -h' works, but 'phablet-flash -h cdimage-touch' doesn't.
> Also see that when you get a help from a specific group, the default ones are
> not part of the list (like -s and -D).

Done, it's no imported as a parent to each command, revno 186 and 187

> - Would it be possible to add a man page for phablet-flash as well?

Can you add this as a bug so I can work on?

> - Please add a way for it to display the license when flashing a community
> based port. If possible I'd make this a requirement for a community build, as
> tag the stamp differently per device/port, so people know what they are
> flashing.

Ok, preliminary support added. revno 188

> - Can you also remove the generated tmp files after flashing the device?
> Check:
> INFO:phablet-flash:Pushing /tmp/tmpPNd_Gv to /cache/recovery/extendedcommand
> ....
> rsalveti@evatp:~$ cat /tmp/tmpPNd_Gv
> mount("/sdcard/");
> install_zip("/sdcard/saucy-preinstalled-touch-armel+manta.zip");
> install_zip("/sdcard/saucy-preinstalled-touch-armhf.zip");

revno 189 deletes the file.

> Code:
> 721 + parser.add_argument('-d', '--device', required=True,
> 722 + help='Specify device to flash')
>
> Mind adding a better help statement here? You could also link at a wiki or
> somewhere explaining how people can look for the supported devices. Later on
> we could also have a --list option to show all the possible devices to flash.

added a link to http://..../Devices (not full link to anchor as it may be an unstable link) in revno 190

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:188
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/129/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/72
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/72
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/72

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/129/rebuild

review: Approve (continuous-integration)

[Sergio Schvezov (sergiusens)]

> 1186 + log.info('Download directory set to %s' % download_dir)
> 1187 + if not os.path.exists(download_dir):
> 1188 + log.info('Creating %s' % download_dir)
> 1189 + os.makedirs(download_dir)
> This is still going to suffer from the makedirs race described in
> https://bugs.launchpad.net/phablet-tools/+bug/1209408
>
> Otherwise, looks good.

I'll fix this in the next MR.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:190
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/130/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/73
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/73
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/73

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/130/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:191
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/131/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/74
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/74
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/74

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/131/rebuild

review: Approve (continuous-integration)

[Ricardo Salveti (rsalveti)]

Looks good, thanks!

review: Approve

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
Approved revid is not set in launchpad. This is most likely a launchpad issue and re-approve should fix it. There is also a chance (although a very small one) this is a permission problem of the ps-jenkins bot.
http://jenkins.qa.ubuntu.com/job/phablet-tools-autolanding/52/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-autolanding/27
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-autolanding/27
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-autolanding/27

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/phablet-tools-autolanding/53/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-autolanding/28
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-autolanding/28
    FAILURE: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-autolanding/28/console

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

review: Approve (continuous-integration)

=== modified file 'debian/changelog'
--- debian/changelog	2013-08-06 00:01:55 +0000
+++ debian/changelog	2013-08-09 01:41:37 +0000
@@ -1,3 +1,12 @@
+phablet-tools (1.0-0ubuntu1) UNRELEASED; urgency=low
+
+  * phablet-flash:
+    - Refactoring to accommodate multiple provisioning scenarios.
+    - Command line switched into using positional arguments.
+    - Adding support to flash community devices.
+
+ -- Sergio Schvezov <[e-mail]>  Thu, 08 Aug 2013 08:19:54 -0300
+
 phablet-tools (0.15+13.10.20130806-0ubuntu1) saucy; urgency=low
 
   [ Andy Doan ]

=== modified file 'debian/control'
--- debian/control	2013-07-14 18:46:18 +0000
+++ debian/control	2013-08-09 01:41:37 +0000
@@ -25,6 +25,7 @@
 Depends:
  android-tools-adb (>= 4.2.2),
  android-tools-fastboot (>= 4.2.2),
+ bzr,
  curl,
  python-configobj,
  python-launchpadlib,

=== modified file 'phablet-flash'
--- phablet-flash	2013-07-30 17:17:05 +0000
+++ phablet-flash	2013-08-09 01:41:37 +0000
@@ -1,479 +1,64 @@
 #! /usr/bin/python2.7
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the the GNU General Public License version 3, as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the applicable version of the GNU Lesser General Public
-# License for more details.
-#.
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-# Copyright (C) 2013 Canonical, Ltd.
 
-import argparse
 import logging
 import os
-import re
-import requests
-import subprocess
-import tempfile
-import gzip
-from os import path
-from time import sleep
+import sys
+
 from phabletutils.device import (AndroidBridge, Fastboot)
-from phabletutils import downloads
-from phabletutils import environment
-from phabletutils import cdimage
+from phabletutils import arguments
+from phabletutils import license
 from phabletutils import settings
-from phabletutils import ubuntuimage
 
-logging.basicConfig(level=logging.INFO, format='%(message)s')
+logging.basicConfig(level=logging.INFO)
 log = logging.getLogger()
-log.name = 'phablet-deploy'
-
-
-def parse_arguments():
-    '''Parses arguments passed in to script.'''
-    parser = argparse.ArgumentParser(
-        description='''phablet flash tool.
-                       Grabs build from the network and deploys to device.
-                       Does best effort to deploy in different ways.''')
-    parser.add_argument('-d',
-                        '--device',
-                        help='''Target device to deploy.''',
-                        required=False,
-                        choices=settings.supported_devices,
-                       )
-    parser.add_argument('-s',
-                        '--serial',
-                        help='''Device serial. Use when more than
-                                one device is connected.''',
-                       )
-    parser.add_argument('--alternate-settings',
-                        help='''Alternate default settings (not common)''',
-                       )
-    parser.add_argument('--no-device-validate',
-                        action='store_true',
-                        default=False,
-                        help='''Skip device validation, use at risk.''',
-                       )
-    parser.add_argument('-b',
-                       '--bootstrap',
-                       help='''Bootstrap the target device, this only
-                               works on Nexus devices or devices that
-                               use fastboot and are unlocked. All user
-                               data is destroyed''',
-                       action='store_true',
-                      )
-    parser.add_argument('-D',
-                       '--download-only',
-                       help='''Download image only, but do not flash device.
-                               Use -d to override target device autodetection
-                               if target device is not connected.''',
-                       action='store_true',
-                      )
-    parser.add_argument('--wipe',
-                       help='''Cleans up all data.''',
-                       action='store_true',
-                      )
-    parser.add_argument('--legacy',
-                       action='store_true',
-                       default=False,
-                       help='''Installs the legacy images'''
-                       )
-    parser.add_argument('--list-revisions',
-                        action='store_true',
-                        required=False,
-                        default=False,
-                        help='List available revisions on cdimage and exits',
-                       )
-    parser.add_argument('--series',
-                        required=False,
-                        default=None,
-                        help='Forces a series, generally not needed',
-                       )
-    group = parser.add_mutually_exclusive_group()
-    group.add_argument('-r',
-                       '--revision',
-                       required=False,
-                       default=None,
-                       help='''Choose a specific release to install from 
-                               cdimage, the format is [series]/[rev].
-                               However for ubuntu-bootstrap it's a 
-                               relative number 0 being latest -1 being
-                               the previous version and so on''',
-                       )
-    group.add_argument('-l',
-                       '--latest-revision',
-                       action='store_true',
-                       required=False,
-                       help='''Pulls the latest tagged revision.''',
-                      )
-    group.add_argument('-p',
-                       '--base-path',
-                       required=False,
-                       default=None,
-                       help='''Installs from base path, you must have the
-                               same directory structure as if you downloaded
-                               for real.
-                               This option is completely offline.'''
-                      )
-    group.add_argument('-u',
-                       '--uri',
-                       required=False,
-                       help='Alternate download uri',
-                      )
-    group.add_argument('--pending',
-                       action='store_true',
-                       required=False,
-                       help='Get pending link from cdimage',
-                      )
-    parser.add_argument('--ubuntu-bootstrap',
-                        action='store_true',
-                        required=False,
-                        help='''Flash the image based upgrade Ubuntu system.
-                                This action wipes the system'''
-                       )
-    return parser.parse_args()
-
-
-# Creates a pathname for user's answer. Touch this file.
+log.name = 'phablet-flash'
+
+
 def accepted_pathname():
     return os.path.expanduser(settings.accept_path)
 
 
-def accepted(pathname):
-    '''
-    Remember that the user accepted the license.
-    '''
-    open(pathname, 'w').close()
-
-
-def has_accepted(pathname):
-    '''
-    Return True iff the user accepted the license once.
-    '''
-    return os.path.exists(pathname)
-
-
-def query(message):
-    '''Display end user agreement to continue with deployment.'''
-    try:
-        while True:
-            print message
-            print 'Do you accept? [yes|no]'
-            answer = raw_input().lower()
-            if answer == 'yes':
-                accepted(accepted_pathname())
-                return True
-            elif answer == 'no':
-                return False
-    except KeyboardInterrupt:
-        log.error('Interruption detected, cancelling install')
-        return False
-
-
-def setup_download_directory(download_dir):
-    '''
-    Tries to create the download directory from XDG_DOWNLOAD_DIR or sets
-    an alternative one.
-
-    Returns path to directory
-    '''
-    log.info('Download directory set to %s' % download_dir)
-    if not os.path.exists(download_dir):
-        log.info('Creating %s' % download_dir)
-        os.makedirs(download_dir)
-
-
-def adb_errors(f):
-    '''Decorating adb error management.'''
-    def _adb_errors(*args, **kwargs):
-        try:
-            return f(*args, **kwargs)
-        except subprocess.CalledProcessError as e:
-            log.error('Error while executing %s' %
-                      e.cmd)
-            log.info('Make sure the device is connected and viewable '
-                    'by running \'adb devices\'')
-            log.info('Ensure you have a root device, one which running '
-                    '\'adb root\' does not return an error')
-            exit(1)
-    return _adb_errors
-
-
-@adb_errors
-def create_recovery_file(adb, device_img, ubuntu_file):
-    template = settings.recovery_script_template
-    # Setup recovery rules
-    recovery_file = tempfile.NamedTemporaryFile(delete=False)
-    # Find out version
-    current_version = adb.getprop('ro.modversion')
-    if current_version.startswith('10.1'):
-        log.debug('Updating from multi user setup')
-        sdcard_path = '/sdcard/0'
-    else:
-        log.info('Updating from non multi user setup')
-        sdcard_path = '/sdcard'
-    recovery_script = template.format(
-        sdcard_path,
-        path.basename(device_img),
-        path.basename(ubuntu_file))
-    with recovery_file as output_file:
-        output_file.write(recovery_script)
-    log.info('Setting up recovery rules')
-    return recovery_file.name
-
-
-def create_ubuntu_command_file(cache_dir, files):
-    ubuntu_command_path = os.path.join(cache_dir, 'ubuntu_command')
-    with open(ubuntu_command_path, 'w+') as output_file:
-        output_file.write(settings.ubuntu_recovery_script)
-        for f in files:
-            filename = os.path.basename(f['filename'])
-            signame = os.path.basename(f['signame'])
-            output_file.write('update %s %s\n' % (filename, signame))
-        output_file.write('unmount system\n')
-    return ubuntu_command_path
-
-
-@adb_errors
-def autodeploy(adb, artifact):
-    ''' Pushes artifact to devices sdcard and deploys'''
-    if not artifact:
-        return
-    # Can't wait-for-device here
-    sleep(15)
-    wipe_device(adb)
-    adb.push(artifact, '/sdcard/autodeploy.zip')
-    log.info('Deploying Ubuntu')
-    adb.reboot(recovery=True)
-    log.info('Installation will complete soon and reboot into Ubuntu')
-
-
-def gunzip(file_path):
-    if not file_path.endswith('.gz'):
-        return file_path
-    target_path = file_path[:-3]
-    log.info('Decompressing %s' % file_path)
-    with open(target_path, 'wb') as target_file:
-        with gzip.open(file_path, 'r') as gzip_file:
-            for chunk in gzip_file.read():
-                target_file.write(chunk)
-    return target_path
-        
-
-def wipe_device(adb):
-    log.info('Clearing /data and /cache')
-    adb.shell('rm -Rf /cache/* /data/* /data/.developer_mode')
-    adb.shell('mkdir /cache/recovery')
-    adb.shell('mkdir /data/media')
-
-
-@adb_errors
-def deploy_recovery_image(adb, device_zip, ubuntu_zip, recovery_file,
-                          wipe=False):
-    ''''
-    Deploys recovery files, recovery script and then reboots to install.
-    '''
-    if wipe:
-        wipe_device(adb)
-    adb.push(device_zip, '/sdcard/')
-    adb.push(ubuntu_zip, '/sdcard/')
-    adb.push(recovery_file, '/cache/recovery/command')
-    adb.reboot(recovery=True)
-    log.info('Once completed the device should reboot into Ubuntu')
-
-
-@adb_errors
-def detect_device(adb, device=None):
-    '''If no argument passed determine them from the connected device.'''
-    # Check CyanogenMod property
-    if not device:
-        device = adb.getprop('ro.cm.device').strip()
-    # Check Android property
-    if not device:
-        device = adb.getprop('ro.product.device').strip()
-    log.info('Device detected as %s' % device)
-    # property may not exist or we may not map it yet
-    if device not in settings.supported_devices:
-        log.error('Unsupported device, autodetect fails device')
-        log.info('When working on flipped images, detection does not '
-                 'work and would require -d')
-        exit(1)
-    return device
-
-
-@adb_errors
-def bootstrap(adb, fastboot, system_img, boot_img, recovery_img):
-    '''
-    Deploys device file using fastboot and launches recovery for ubuntu
-    deployment.
-    '''
-    adb.reboot(bootloader=True)
-    log.warning('The device needs to be unlocked for the following to work')
-
-    fastboot.flash('system', gunzip(system_img))
-    fastboot.flash('boot', boot_img)
-    if recovery_img:
-        fastboot.flash('recovery', recovery_img)
-        fastboot.boot(recovery_img)
-    else:
-        log.info('Successfully flashed, now rebooting device')
-        fastboot.reboot()
-
-
-@adb_errors
-def validate_device(adb):
-    '''
-    Validates if the image would be installable on the target
-    '''
-    df = adb.shell('df -h').split('\n')
-    try:
-        free_data = map(str.split,
-                        filter(lambda s: '/data' in s, df))[0][2]
-    except IndexError:
-        log.error('Cannot find /data mountpoint')
-        adb.reboot()
-        exit(1)
-    if free_data[-1:] == 'G' and float(free_data[:-1]) >= 3:
-        log.info('Storage requirements in /data satisfied')
-    else:
-        log.error('Not enough space in /data, found %s, rebooting', free_data)
-        adb.reboot()
-        exit(1)
-
-
-@adb_errors
-def setup_ubuntu_image_update(device, revision=-1):
-    '''
-    Sets up as described in https://wiki.ubuntu.com/ImageBasedUpgrades.
-    '''
-    # This is temporary until more commonality is found.
-    json_latest = ubuntuimage.get_json_from_index(device, revision)
-    download_dir = environment.get_download_dir_full_path(
-        os.path.join(settings.download_dir,'imageupdates',
-            str(json_latest['version'])))
-    setup_download_directory(download_dir)
-    files = ubuntuimage.download_images(download_dir, json_latest)
-    ubuntu_command_path = create_ubuntu_command_file(download_dir,
-        files['updates'])
-    return files, ubuntu_command_path
-
-
-def deploy_ubuntu_image_update(adb, fastboot, recovery_img, files, 
-                               ubuntu_command_path):
-    adb.reboot(recovery=True)
-    wipe_device(adb)
-    for key in files:
-        for entry in files[key]:
-            adb.push(entry['filename'], '/cache/recovery/')
-            adb.push(entry['signame'], '/cache/recovery/')
-    adb.push(ubuntu_command_path, '/cache/recovery/ubuntu_command')
-    adb.reboot(bootloader=True)
-    fastboot.flash('recovery', recovery_img)
-    fastboot.boot(recovery_img)
-
-
-def main(args):
-    if args.legacy and args.pending:
-        log.error('Cannot use legacy and pending together')
-        exit(1)
-    if args.ubuntu_bootstrap and args.revision:
-        ubuntu_revision = int(args.revision) - 1
-        if ubuntu_revision >= 0:
-            log.error('Revision must be 0 for current or negative number')
-            exit(1)
-        args.revision = None
-    else:
-        ubuntu_revision = -1
-    if args.list_revisions:
-        # Easy hack to get rid of the logger and inhibit requests from logging
-        log.setLevel(logging.FATAL)
-        env = environment.Environment(args.uri, None, args.series, 
-            args.latest_revision, args.revision, args.pending,  args.legacy, 
-            args.base_path, settings)
-        env.project.list_revisions()
-        exit(0)
-    if not has_accepted(accepted_pathname()) and \
-       not query(settings.legal_notice):
-        exit(1)
-    adb = AndroidBridge(args.serial)
-    fastboot = Fastboot(args.serial)
-    # Initializes the adb server if it is not running
-    adb.start()
-    device = detect_device(adb, args.device)
-    try:
-        env = environment.Environment(args.uri, device, args.series, 
-            args.latest_revision, args.revision, args.pending,  args.legacy, 
-            args.base_path, settings)
-    except EnvironmentError as e:
-        log.error(e.message)
-        exit(1)
-    log.info('Download uri set to %s' % env.download_uri)
-    if env.download_uri:
-        setup_download_directory(env.download_dir)
-    download_list = []
-    if args.ubuntu_bootstrap:
-        download_list.append(env.recovery_img_path)
-    elif args.bootstrap:
-        [download_list.append(f) for f in env.bootstrap_files]
-    else:
-        [download_list.append(f) for f in env.recovery_files]
-    try:
-        if env.project:
-            hashes = env.project.hashes
-        else:
-            hashes = None
-        log.info('Retrieving files')
-        downloader = downloads.Downloader(env.download_uri,
-            download_list, hashes)
-        downloader.download()
-    except KeyboardInterrupt:
-        log.info('To continue downloading in the future, rerun the same '
-                 'command')
-        exit(1)
-    except EnvironmentError as e:
-        log.error(e.message)
-        exit(1)
-    except subprocess.CalledProcessError:
-        log.error('Error while downloading, ensure connection')
-        exit(1)
-    if env.download_uri and env.project:
-        env.store_hashes()
-    if not args.download_only:
-        if args.ubuntu_bootstrap:
-            ubuntu_files, ubuntu_command_path = \
-                setup_ubuntu_image_update(device, ubuntu_revision) 
-            deploy_ubuntu_image_update(adb, fastboot, env.recovery_img_path,
-                ubuntu_files, ubuntu_command_path)
-        elif args.bootstrap:
-            bootstrap(adb, fastboot, env.system_img_path,
-                env.boot_img_path, env.recovery_img_path)
-            autodeploy(adb, env.ubuntu_zip_path)
-        else:
-            adb.reboot(recovery=True)
-            validate_device(adb)
-            recovery_file = create_recovery_file(adb, 
-                env.device_zip_path, env.ubuntu_zip_path)
-            deploy_recovery_image(adb, env.device_zip_path,
-                env.ubuntu_zip_path, recovery_file, args.wipe)
-
-
-def import_alt_settings(alternate_settings):
-    import imp
-    global settings
-    dirname, basename = os.path.split(alternate_settings)
-    f, filename, desc = imp.find_module(basename.rstrip('\.py'), [dirname])
-    settings = imp.load_module(basename.rstrip('\.py'), f, filename, desc)
+def main(argv):
+    parser = arguments.get_parser()
+    args = parser.parse_args(argv[1:])
+    if args.debug:
+        log.setLevel(logging.DEBUG)
+    if not license.has_accepted(accepted_pathname()) and \
+       not license.query(settings.legal_notice, accepted_pathname()):
+        exit(1)
+    try:
+        project = args.func(args)
+        if project:
+            fastboot = Fastboot(args.serial)
+            adb = AndroidBridge(args.serial)
+            adb.start()
+            project.download()
+            if not args.download_only:
+                project.install(adb, fastboot)
+    except KeyboardInterrupt:
+        log.info('Provisioning manually interrupted. Resume by rerunning '
+                 'the command')
+        exit(1)
+    except Exception as e:
+        log.error(e)
+        if args.debug:
+            log.exception(e)
+        exit(1)
 
 
 if __name__ == "__main__":
-    args = parse_arguments()
-    if args.alternate_settings:
-        import_alt_settings(args.alternate_settings)
-    main(args)
+    main(sys.argv)

=== modified file 'phabletutils/__init__.py'
--- phabletutils/__init__.py	2013-01-16 14:55:09 +0000
+++ phabletutils/__init__.py	2013-08-09 01:41:37 +0000
@@ -1,5 +1,6 @@
 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
 # Copyright 2013 Canonical
+# Author: Sergio Schvezov <[e-mail]>
 #
 # This program is free software: you can redistribute it and/or modify it
 # under the terms of the GNU General Public License version 3, as published

=== added file 'phabletutils/arguments.py'
--- phabletutils/arguments.py	1970-01-01 00:00:00 +0000
+++ phabletutils/arguments.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,256 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import argparse
+import logging
+import tempfile
+import urllib
+import urlparse
+
+from phabletutils import cdimage
+from phabletutils import environment
+from phabletutils import resources
+from phabletutils import settings
+
+log = logging.getLogger()
+
+
+class PathAction(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        log.debug('PathAction: %r %r %r' %
+                  (namespace, values, option_string))
+        uri = urlparse.urlparse(values)
+        if not uri.scheme or uri.scheme == 'file':
+            if '%' in uri.path:
+                zip_file_path = urllib.unquote(
+                    urlparse.urlparse(uri.path))
+            else:
+                zip_file_path = uri.path
+            zip_file_uri = None
+            check = False
+        elif uri.scheme == 'http' or uri.scheme == 'https':
+            zip_file_path = tempfile.mktemp()
+            zip_file_uri = values
+            check = True
+        log.debug('Download from %s, path on disk %s' %
+                  (zip_file_uri, zip_file_path))
+        artifact = resources.File(file_path=zip_file_path,
+                                  file_uri=zip_file_uri,
+                                  check=check)
+        setattr(namespace, self.dest, artifact)
+
+
+class RevisionListAction(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        log.debug('RevisionListAction: %r %r %r' %
+                  (namespace, values, option_string))
+        project = 'ubuntu-touch-preview'
+        uri = '%s/%s' % (settings.cdimage_uri_base, project)
+        setattr(namespace, 'func', environment.list_revisions)
+        setattr(namespace, 'uri', uri)
+
+
+class RevisionAction(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        log.debug('RevisionAction: %r %r %r' %
+                  (namespace, values, option_string))
+        project = 'ubuntu-touch-preview'
+        base_uri = '%s/%s' % (settings.cdimage_uri_base, project)
+        if values:
+            revision_split = values.split('/')
+            if len(revision_split) != 2:
+                raise EnvironmentError(
+                    'Improper use of revision, needs to be formatted like'
+                    '[series]/[revision]. Use --list-revisions to find'
+                    'the available revisions on cdimage')
+            series = revision_split[0]
+            build = revision_split[1]
+        else:
+            series, build = cdimage.get_latest_revision(base_uri)
+        uri = '%s/%s/%s' % (base_uri, series, build)
+        setattr(namespace, self.dest, True)
+        setattr(namespace, 'series', series)
+        setattr(namespace, 'build', build)
+        setattr(namespace, 'uri', uri)
+
+
+def cdimage_touch(parent_parser, parents):
+    parser = parent_parser.add_parser(
+        'cdimage-touch', parents=parents,
+        help='Provisions the device with a CDimage build of Ubuntu Touch.')
+    parser.set_defaults(func=environment.setup_cdimage_touch,
+                        project='ubuntu-touch',
+                        series=settings.default_series,
+                        build=None,
+                        uri=None)
+    parser.add_argument('-b',
+                        '--bootstrap',
+                        help='''Bootstrap the target device, this only
+                                works on Nexus devices or devices that
+                                use fastboot and are unlocked. All user
+                                data is destroyed''',
+                        action='store_true')
+    group = parser.add_mutually_exclusive_group()
+    group.add_argument('--pending',
+                       action='store_true',
+                       required=False,
+                       help='Get pending link from cdimage')
+    group.add_argument('-p',
+                       '--base-path',
+                       required=False,
+                       default=None,
+                       help='''Installs from base path, you must have
+                               the same directory structure as if you
+                               downloaded for real.
+                               This option is completely offline.''')
+    return parser
+
+
+def ubuntu_system(parent_parser, parents):
+    parser = parent_parser.add_parser(
+        'ubuntu-system', parents=parents,
+        help='Provisions the device with an Ubuntu Image Based Upgrade image.')
+    parser.set_defaults(func=environment.setup_ubuntu_system,
+                        revision=0,
+                        series=settings.default_series,
+                        project='imageupdates')
+    parser.add_argument('--revision',
+                        type=int,
+                        help='''Download a relative revision from current
+                                (-1, -2, ...) or a specific version.''')
+    return parser
+
+
+def legacy(parent_parser, parents):
+    parser = parent_parser.add_parser(
+        'cdimage-legacy', parents=parents,
+        help='Provisions the device with legacy unflipped images.')
+    parser.set_defaults(func=environment.setup_cdimage_legacy,
+                        project='ubuntu-touch-preview',
+                        series=settings.default_series,
+                        build=None,
+                        uri=None)
+    parser.add_argument('-b',
+                        '--bootstrap',
+                        help='''Bootstrap the target device, this only
+                                works on Nexus devices or devices that
+                                use fastboot and are unlocked. All user
+                                data is destroyed''',
+                        action='store_true')
+    group = parser.add_mutually_exclusive_group()
+    group.add_argument('--list-revisions',
+                       action=RevisionListAction,
+                       nargs=0,
+                       help='List available revisions on cdimage and exits')
+    group.add_argument('-r',
+                       '--revision',
+                       action=RevisionAction,
+                       help='''Choose a specific release to install
+                               from cdimage, the format is
+                               [series]/[rev].''')
+    group.add_argument('-l',
+                       '--latest-revision',
+                       action=RevisionAction,
+                       nargs=0,
+                       help='''Pulls the latest tagged revision.''')
+    group.add_argument('-p',
+                       '--base-path',
+                       required=False,
+                       default=None,
+                       help='''Installs from base path, you must have
+                               the same directory structure as if you
+                               downloaded for real.
+                               This option is completely offline.''')
+    return parser
+
+
+def community(parent_parser, parents):
+    parser = parent_parser.add_parser(
+        'community', parents=parents,
+        help='Provisions the device with a community supported build.')
+    parser.add_argument('-d', '--device', required=True,
+                        help='''Specify device to flash.
+                                Find out more about flashable devices at
+                                https://wiki.ubuntu.com/Touch/Devices''')
+    parser.set_defaults(func=environment.setup_community,
+                        series=settings.default_series)
+    return parser
+
+
+def common_non_system():
+    parser = argparse.ArgumentParser(add_help=False)
+    parser.add_argument('--device-path',
+                        action=PathAction,
+                        help='''uri to device zip to flash,
+                                e.g.; file:///..., http://.''',)
+    parser.add_argument('--ubuntu-path',
+                        action=PathAction,
+                        help='''uri to ubuntu zip to flash,
+                                e.g.; file:///..., http://.''',)
+    parser.add_argument('--wipe',
+                        action='store_true',
+                        help='''Wipes device data.''')
+    return parser
+
+
+def common_supported():
+    parser = argparse.ArgumentParser(add_help=False)
+    parser.add_argument('-d',
+                        '--device',
+                        help='''Target device to deploy.''',
+                        required=False,
+                        choices=settings.supported_devices)
+    return parser
+
+
+def common():
+    parser = argparse.ArgumentParser(add_help=False)
+    parser.add_argument('--debug',
+                        action='store_true',
+                        help='''Enable debug messages.''')
+    parser.add_argument('-s',
+                        '--serial',
+                        help='''Device serial. Use when more than
+                                one device is connected.''')
+    parser.add_argument('-D',
+                        '--download-only',
+                        action='store_true',
+                        help='Download image only, but do not flash device.')
+    return parser
+
+
+def get_parser():
+    """
+    Returns a Namespace of the parsed arguments from the command line.
+    """
+    parser = argparse.ArgumentParser(
+
+        description='''phablet-flash is used to provision devices.
+                       It does a best effort to deploy in different ways.''',
+        epilog='''Use -h or --help after each command to learn about
+                  their provisioning options.''')
+    # Parsers
+    common_parser = common()
+    common_supported_parser = common_supported()
+    common_non_system_parser = common_non_system()
+    sub = parser.add_subparsers(title='Commands', metavar='')
+    cdimage_touch(sub, [common_parser, common_supported_parser,
+                        common_non_system_parser])
+    legacy(sub, [common_parser, common_supported_parser,
+                 common_non_system_parser])
+    ubuntu_system(sub, [common_supported_parser, ])
+    community(sub, [common_parser, common_non_system_parser, ])
+    return parser

=== modified file 'phabletutils/cdimage.py'
--- phabletutils/cdimage.py	2013-07-11 22:28:31 +0000
+++ phabletutils/cdimage.py	2013-08-09 01:41:37 +0000
@@ -1,24 +1,29 @@
 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
-# Copyright 2013 Canonical
-#
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the GNU General Public License version 3, as published
-# by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the GNU General Public License for more details.
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
 
-# You should have received a copy of the GNU General Public License along
-# with this program.  If not, see <http://www.gnu.org/licenses/>.
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import logging
 import re
 import requests
+import os.path
 import subprocess
 import urlparse
 
+from phabletutils import hashes
+from phabletutils import resources
+from phabletutils import settings
 
 log = logging.getLogger()
 
@@ -95,8 +100,6 @@
             print 'No releases for %s available' % rev['release']
     if not revisions:
         print 'No revisions have been tagged for this project yet'
-        print 'To list revisions from the preview images add the ' \
-              '--legacy switch'
 
 
 def _get_link_target(uri):
@@ -107,40 +110,27 @@
     return link
 
 
-def get_latest_current(cdimage_uri):
+def get_build(cdimage_uri, pending=False):
     '''Returns the latest build in current.'''
-    uri = '%s/%s' % (cdimage_uri, 'current')
-    build = _get_link_target(uri)
-    return build
-
-
-def get_latest_pending(cdimage_uri):
-    '''Returns the latest build in pending.'''
-    uri = '%s/%s' % (cdimage_uri, 'pending')
-    build = _get_link_target(uri)
-    return build
-
-
-def get_sha256_dict(hash_file_content):
-    '''Returns a dictionary with the sha256 sums for all files.'''
-    if not hash_file_content:
-        log.debug('hash file is empty')
-        return None
-    hash_list = filter((lambda x: len(x) is not 0),
-                       hash_file_content.split('\n'))
-    hash_list = [h.split() for h in hash_list]
-    hash_dict = {}
-    for hash_entry in hash_list:
-        if hash_entry[1][0] == '*':
-            hash_entry[1] = hash_entry[1][1:]
-        hash_dict[hash_entry[1]] = hash_entry[0]
-    return hash_dict
-
-
-def get_sha256_content(cdimage_hash_uri):
-    '''Fetches the SHA256 sum file from cdimage.'''
-    hash_request = requests.get(cdimage_hash_uri)
-    if hash_request.status_code != 200:
-        return None
-    else:
-        return hash_request.content
+    if pending:
+        uri = '%s/%s' % (cdimage_uri, 'pending')
+    else:
+        uri = '%s/%s' % (cdimage_uri, 'current')
+    build = _get_link_target(uri)
+    return build
+
+
+def get_file(file_key, series, download_dir, project='ubuntu-touch',
+             device=None):
+    uri = '%s/%s/daily-preinstalled/current' % (
+        settings.cdimage_uri_base, project)
+    hash_dict = hashes.load_hash(uri, 'SHA256SUMS')
+    if device:
+        file_name = settings.files_arch_any[project][file_key] %\
+            (series, device)
+    else:
+        file_name = settings.files_arch_all[project][file_key] % series
+    return resources.File(
+        file_path=os.path.join(download_dir, file_name),
+        file_uri='%s/%s' % (uri, file_name),
+        file_hash=hash_dict[file_name])

=== added file 'phabletutils/community.py'
--- phabletutils/community.py	1970-01-01 00:00:00 +0000
+++ phabletutils/community.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,129 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import print_function
+
+import json
+import hashlib
+import logging
+import os
+import os.path
+import subprocess
+
+from phabletutils import cdimage
+from phabletutils import downloads
+from phabletutils import resources
+from phabletutils import license
+from phabletutils import settings
+
+log = logging.getLogger()
+branch_template = 'lp:~{0}-image-dev/phablet-image-info/{0}'
+base_dir = os.path.join(settings.download_dir, 'community')
+
+license_template = '''This community project has the following license:
+
+{0}
+'''
+
+
+def branch_project(device):
+    branch = branch_template.format(device.replace('_', '-'))
+    log.info('Obtaining project branch from %s' % branch)
+    download_dir = downloads.get_full_path(os.path.join(base_dir, device))
+    target = os.path.join(download_dir, 'config')
+    if os.path.exists(target):
+        subprocess.check_call(['bzr', 'update'], cwd=target)
+    else:
+        subprocess.check_call(
+            ['bzr', 'checkout', '--lightweight', branch, target])
+    log.info('Target config retrieved to %s' % target)
+    # Best time to do this is right after retrieving the config
+    ensure_license_accept(download_dir, os.path.join(target, 'license'))
+    return target
+
+
+def ensure_license_accept(download_dir, license_file):
+    if not os.path.exists(license_file):
+        raise EnvironmentError('Project does not offer a license file')
+    accept_path = os.path.join(download_dir, '.license_accept')
+    with open(license_file, 'r') as f:
+        device_license = f.read()
+    device_license = 'LICENSE TEXT NOT PROVIDED BY PORT MAINTAINER' \
+        if not device_license else device_license
+    message = license_template.format(device_license)
+    if not license.has_accepted(accept_path) and \
+       not license.query(message, accept_path):
+        raise RuntimeError('License not accepted.')
+
+
+def load_manifest(directory):
+    manifest_file = os.path.join(directory, 'manifest.json')
+    if not os.path.exists(manifest_file):
+        raise RuntimeError('Cannot locate %s' % manifest_file)
+    with open(manifest_file) as f:
+        manifest_dict = json.load(f)
+    if 'device' not in manifest_dict:
+        raise EnvironmentError('device entry required in manifest')
+    if 'revision' not in manifest_dict:
+        manifest_dict['revision'] = None
+    if 'storage' not in manifest_dict:
+        manifest_dict['storage'] = '/sdcard/'
+    if 'ubuntu' not in manifest_dict:
+        manifest_dict['ubuntu'] = None
+    log.debug(json.dumps(manifest_dict))
+    return manifest_dict
+
+
+def get_download_dir(device, revision=None):
+    download_dir = os.path.join(settings.download_dir, 'community', device)
+    if revision:
+        download_dir = os.path.join(download_dir, revision)
+    return downloads.get_full_path(download_dir)
+
+
+def get_files(manifest_dict, download_dir, series):
+    files = {}
+    for key in ('device', 'ubuntu'):
+        if isinstance(manifest_dict[key], dict):
+            item = manifest_dict[key]
+            hash_type = item['hash_func'] if 'hash_func' in item else None
+            log.debug('%s has config uri: %s' % (key, item['uri']))
+            files[key] = resources.File(
+                file_path=os.path.join(download_dir, '%s.zip' % key),
+                file_uri=item['uri'],
+                file_hash=item['hash'] if 'hash' in item else None,
+                file_hash_func=get_hash_func(hash_type) if hash_type else None)
+        elif isinstance(manifest_dict[key], str) or \
+                isinstance(manifest_dict[key], unicode):
+            log.debug('%s has config uri: %s' % (key, manifest_dict[key]))
+            files[key] = resources.File(
+                file_path=os.path.join(download_dir, '%s.zip' % key),
+                file_uri=manifest_dict[key],
+                file_hash=None)
+        log.debug('%s is type %s' % (key, type(manifest_dict[key])))
+    if not manifest_dict['ubuntu']:
+        log.warning('Using Ubuntu Touch current build for ubuntu image')
+        files['ubuntu'] = cdimage.get_file(file_key='ubuntu_zip',
+                                           series=series,
+                                           download_dir=download_dir)
+    return files
+
+
+def get_hash_func(hash_type):
+    return {
+        'md5': hashlib.md5,
+        'sha256': hashlib.sha256,
+        }.get(hash_type, None)

=== modified file 'phabletutils/device.py'
--- phabletutils/device.py	2013-07-02 18:37:54 +0000
+++ phabletutils/device.py	2013-08-09 01:41:37 +0000
@@ -1,17 +1,18 @@
 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
-# Copyright 2013 Canonical
-#
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the GNU General Public License version 3, as published
-# by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the GNU General Public License for more details.
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
 
-# You should have received a copy of the GNU General Public License along
-# with this program.  If not, see <http://www.gnu.org/licenses/>.
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import subprocess
 import logging

=== modified file 'phabletutils/downloads.py'
--- phabletutils/downloads.py	2013-08-02 22:01:03 +0000
+++ phabletutils/downloads.py	2013-08-09 01:41:37 +0000
@@ -1,108 +1,130 @@
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the the GNU General Public License version 3, as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the applicable version of the GNU Lesser General Public
-# License for more details.
-#.
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-# Copyright (C) 2013 Canonical, Ltd.
 
+import configobj
 import contextlib
 import fcntl
 import hashlib
 import logging
 import os
+import requests
 import subprocess
 
+from xdg.BaseDirectory import xdg_config_home
+
 
 log = logging.getLogger()
 
 
-def download(uri, target):
-    '''Downloads an artifact into target.'''
-    log.info('Downloading %s' % uri)
-    if uri.startswith('http://cdimage.ubuntu.com') or \
-       uri.startswith('https://system-image.ubuntu.com'):
-        subprocess.check_call(['wget', '-c', uri, '-O', target])
-    else:
-        subprocess.check_call(['curl', '-L', '-C', '-', uri, '-o', target])
-
-
 @contextlib.contextmanager
 def flocked(lockfile):
     lockfile += '.lock'
     with open(lockfile, 'w') as f:
-        log.debug('aquiring lock for %s', lockfile)
+        log.debug('Aquiring lock for %s', lockfile)
         try:
             fcntl.lockf(f, fcntl.LOCK_EX)
             yield
         finally:
+            log.debug('Releasing lock for %s', lockfile)
             fcntl.lockf(f, fcntl.LOCK_UN)
 
 
-class Downloader(object):
-    '''A helper to fetch multiple artifacts.'''
-
-    def __init__(self, uri, download_list, hashes):
-        '''Initializes a download helper for android based images.'''
-        self._uri = uri
-        self._download_list = download_list
-        self._hashes = hashes
-        if hashes:
-            self.validate = self._validate
-        else:
-            self.validate = self._validate_legacy
-
-    def download(self):
-        '''Downloads and validates the download list.'''
-        for file_path in self._download_list:
-            with flocked(file_path):
-                if self.validate(file_path):
-                    continue
-                uri = '%s/%s' % (self._uri, os.path.basename(file_path))
-                if not uri:
-                    fmt = ('In offline mode and checksum does not match '
-                           'for at least %s')
-                    raise EnvironmentError(fmt % file_path)
-                download(uri, file_path)
-                if not self._hashes:
-                    file_hash = '%s.md5sum'
-                    download(file_hash % uri, file_hash % file_path)
-                if not self.validate(file_path):
-                    fmt = 'File download failed for %s to %s'
-                    raise EnvironmentError(fmt % (uri, file_path))
-                log.info('Validating download of %s' % file_path)
-
-    def _validate(self, file_path):
-        '''Validates downloaded files against a checksum.'''
-        file_name = os.path.basename(file_path)
-        hashes = self._hashes()
-        if file_name not in hashes:
-            return False
-        return self.checksum_file(file_path) == hashes[file_name]
-
-    def _validate_legacy(self, file_path):
-        '''Validates with the legacy method of individual md5sum files.'''
-        file_hash = '%s.md5sum' % file_path
-        if not os.path.exists(file_hash):
-            return False
-        with open(file_hash, 'r') as f:
-            read_hash = f.read().split()[0]
-        return self.checksum_file(file_path, hashlib.md5) == read_hash
-
-    def checksum_file(self, file_path, sum_method=hashlib.sha256):
-        '''Returns the checksum for a file with a specified algorightm.'''
-        file_sum = sum_method()
-        if not os.path.exists(file_path):
-            return None
-        with open(file_path, 'rb') as f:
-            for file_chunk in iter(
-                    lambda: f.read(file_sum.block_size * 128), b''):
-                file_sum.update(file_chunk)
-        return file_sum.hexdigest()
+def setup_download_directory(download_dir):
+    '''
+    Tries to create the download directory from XDG_DOWNLOAD_DIR or sets
+    an alternative one.
+
+    Returns path to directory
+    '''
+    log.info('Download directory set to %s' % download_dir)
+    if not os.path.exists(download_dir):
+        log.info('Creating %s' % download_dir)
+        os.makedirs(download_dir)
+
+
+def get_full_path(subdir):
+    try:
+        userdirs_file = os.path.join(xdg_config_home, 'user-dirs.dirs')
+        userdirs_config = configobj.ConfigObj(userdirs_file, encoding='utf-8')
+        userdirs_download = os.path.expandvars(
+            userdirs_config['XDG_DOWNLOAD_DIR'])
+        download_dir = userdirs_download
+    except KeyError:
+        download_dir = os.path.expandvars('$HOME')
+        log.warning('XDG_DOWNLOAD_DIR could not be read')
+    directory = os.path.join(download_dir, subdir)
+    setup_download_directory(directory)
+    return directory
+
+
+def checksum_verify(file_path, file_hash, sum_method=hashlib.sha256):
+    '''Returns the checksum for a file with a specified algorightm.'''
+    file_sum = sum_method()
+    log.debug('Verifying file: %s against: %s' % (file_path, file_hash))
+    if not os.path.exists(file_path):
+        log.debug('File %s not found' % file_path)
+        return False
+    with open(file_path, 'rb') as f:
+        for file_chunk in iter(
+                lambda: f.read(file_sum.block_size * 128), b''):
+            file_sum.update(file_chunk)
+    if file_hash == file_sum.hexdigest():
+        return True
+    else:
+        log.debug('Calculated sum mismatch calculated %s != %s' %
+                  (file_sum.hexdigest(), file_hash))
+    return False
+
+
+def _download(uri, path):
+    if uri.startswith('http://cdimage.ubuntu.com') or \
+       uri.startswith('https://system-image.ubuntu.com'):
+        subprocess.check_call(['wget',
+                               '-c',
+                               uri,
+                               '-O',
+                               path])
+    else:
+        subprocess.check_call(['curl',
+                               '-L',
+                               '-C',
+                               '-',
+                               uri,
+                               '-o',
+                               path])
+
+
+def download_sig(artifact):
+    '''Downloads an artifact into target.'''
+    log.info('Downloading %s to %s' % (artifact.uri, artifact.path))
+    with flocked(artifact._sig_path):
+        _download(artifact.sig_uri, artifact.sig_path)
+
+
+def download(artifact):
+    '''Downloads an artifact into target.'''
+    log.info('Downloading %s to %s' % (artifact.uri, artifact.path))
+    with flocked(artifact._path):
+        _download(artifact.uri, artifact.path)
+
+
+def get_content(uri):
+    '''Fetches the SHA256 sum file from cdimage.'''
+    content_request = requests.get(uri)
+    if content_request.status_code != 200:
+        return None
+    else:
+        return content_request.content

=== modified file 'phabletutils/environment.py'
--- phabletutils/environment.py	2013-08-02 21:00:48 +0000
+++ phabletutils/environment.py	2013-08-09 01:41:37 +0000
@@ -1,42 +1,39 @@
 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
-# Copyright 2013 Canonical
-#
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the GNU General Public License version 3, as published
-# by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License along
-# with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import configobj
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import print_function
+
+import hashlib
 import logging
+import os.path
 import requests
 
-from os import path
+from phabletutils.device import AndroidBridge
 from phabletutils import cdimage
-from xdg.BaseDirectory import xdg_config_home
+from phabletutils import community
+from phabletutils import downloads
+from phabletutils import hashes
+from phabletutils import resources
+from phabletutils import projects
+from phabletutils import settings
+from phabletutils import ubuntuimage
 
 log = logging.getLogger()
 
 
-def get_download_dir_full_path(subdir):
-    try:
-        userdirs_file = path.join(xdg_config_home, 'user-dirs.dirs')
-        userdirs_config = configobj.ConfigObj(userdirs_file, encoding='utf-8')
-        userdirs_download = path.expandvars(
-            userdirs_config['XDG_DOWNLOAD_DIR'])
-        download_dir = userdirs_download
-    except KeyError:
-        download_dir = path.expandvars('$HOME')
-        log.warning('XDG_DOWNLOAD_DIR could not be read')
-    return path.join(download_dir, subdir)
-
-
 def get_ubuntu_stamp(uri):
     '''Downloads the jenkins build id from stamp file'''
     try:
@@ -54,224 +51,187 @@
         log.error('Could not download build data from jenkins... bailing')
         exit(1)
     except IndexError:
-        log.error('Jenkins data format has changed, incompatible')
-        exit(1)
+        raise EnvironmentError('Jenkins data format has changed, incompatible')
     return jenkins_build_id
 
 
-class Project(object):
-
-    @property
-    def uri(self):
-        return self._uri
-
-    @property
-    def download_uri(self):
-        return self._download_uri
-
-    @property
-    def download_dir(self):
-        return self._download_dir
-
-    @property
-    def name(self):
-        return self._project
-
-    @property
-    def series(self):
-        return self._series
-
-    def hashes(self):
-        return cdimage.get_sha256_dict(self.hash_content)
-
-    @property
-    def hash_content(self):
-        self._load_hashes()
-        return self._hash_content
-
-    @property
-    def hash_file_name(self):
-        return self._hash_file_name
-
-    def list_revisions(self):
-        revisions = cdimage.get_available_revisions(self._uri)
-        cdimage.display_revisions(revisions)
-
-    def __init__(self, project, series, cdimage_uri_base, base_path,
-                 hash_file_name, pending, revision=None, latest_revision=None):
-        if not project:
-            return None
-        log.debug('project: %s, series: %s, cdimage_uri_base: %s, '
-                  'base_path: %s, hash_file_name: %s, revision: %s, '
-                  'latest_revision: %s, pending: %s' % (project, series, 
-                  cdimage_uri_base, base_path, hash_file_name, revision, 
-                  latest_revision, pending))
-        self._project = project
-        self._uri = '%s/%s' % (cdimage_uri_base, project)
-        self._series = series
-        self._hash_file_name = hash_file_name
-
-        if base_path:
-            download_uri = None
-            download_dir = base_path
-        elif latest_revision:
-            self._series, revision = \
-                cdimage.get_latest_revision(self._uri)
-            download_uri = '%s/%s/%s' % (self._uri,
-                                         self._series, revision)
-            download_dir = path.join(project, self._series, revision)
-        elif revision:
-            revision_split = revision.split('/')
-            if len(revision_split) != 2:
-                raise EnvironmentError(
-                    'Improper use of revision, needs to be formatted like'
-                    '[series]/[revision]. Use --list-revisions to find'
-                    'the available revisions on cdimage')
-            # TODO add verification that uri exists
-            self._series = revision_split[0]
-            download_uri = '%s/%s' % (self._uri, revision)
-            download_dir = path.join(project, self._series, revision)
-        else:
-            uri = '%s/daily-preinstalled' % self._uri
-            if pending:
-                link = cdimage.get_latest_pending(uri)
-            else: 
-                link = cdimage.get_latest_current(uri)
-            download_uri = '%s/%s' % (uri, link)
-            download_dir = path.join(project, link)
-        self._download_uri = download_uri
-        self._download_dir = download_dir
-
-    def _load_hashes(self):
-        self._hash_path = path.join(self._download_dir, self._hash_file_name)
-        if path.exists(self._hash_path):
-            with open(self._hash_path, 'r') as f:
-                hash_content = f.read()
-        else:
-            hash_content = cdimage.get_sha256_content(
-                '%s/%s' % (self._download_uri, self._hash_file_name))
-        self._hash_content = hash_content
-
-
-class Environment(object):
-    '''All the hacks to support multiple environments live here.'''
-
-    @property
-    def project(self):
-        return self._project
-
-    @property
-    def download_dir(self):
-        return self._download_dir
-
-    @property
-    def download_uri(self):
-        return self._download_uri
-
-    @property
-    def bootstrap_files(self):
-        return [self._files[i] for i in self._bootstrap_files]
-
-    @property
-    def recovery_files(self):
-        return [self._files[i] for i in self._recovery_files]
-
-    @property
-    def device_zip_path(self):
-        return self._files['device_zip']
-
-    @property
-    def ubuntu_zip_path(self):
-        return self._files['ubuntu_zip']
-
-    @property
-    def system_img_path(self):
-        return self._files['system_img']
-
-    @property
-    def boot_img_path(self):
-        return self._files['boot_img']
-
-    @property
-    def recovery_img_path(self):
-        return self._files['recovery_img']
-
-    def __init__(self, preset_uri, device, series, latest_revision, revision,
-                 pending, legacy, base_path, settings):
-        self._set_project(series, pending, legacy, base_path, latest_revision,
-                          revision, settings)
-        self._set_download(preset_uri, settings)
-        self._files, self._bootstrap_files, self._recovery_files = \
-            self._set_files(device, settings)
-
-    def _set_project(self, series, pending, legacy, base_path,
-                     latest_revision, revision, settings):
-        if getattr(settings, 'revision', None):
-            if not series:
-                series = settings.default_series
-            if legacy:
-                project_name = 'ubuntu-touch-preview'
-            else:
-                project_name = 'ubuntu-touch'
-            self._project = Project(
-                project_name, series, settings.cdimage_uri_base, base_path,
-                settings.hash_file_name, pending, revision, latest_revision)
-        else:
-            self._project = None
-
-    def _set_download(self, preset_uri, settings):
-        if preset_uri:
-            self._download_uri = preset_uri
-        elif not self._project:
-            self._download_uri = settings.download_uri
-        elif self._project:
-            self._download_uri = self._project.download_uri
-        else:
-            raise EnvironmentError('Environment not setup correctly')
-        if self._project:
-            download_dir = path.join(settings.download_dir,
-                                     self._project.download_dir)
-        else:
-            download_dir = path.join(settings.download_dir,
-                                     get_ubuntu_stamp(self.download_uri))
-        log.debug('Download set to %s' % self.download_uri)
-        if self._download_uri:
-            self._download_dir = get_download_dir_full_path(download_dir)
-        else:
-            self._download_dir = download_dir
-
-    def _set_files(self, device, settings):
-        files = {}
-        project = self._project
-        if project:
-            templ = settings.files[project.name]
-            files['ubuntu_zip'] = templ['ubuntu_zip'] % (project.series,)
-            for key in templ:
-                if key is 'ubuntu_zip':
-                    continue
-                files[key] = templ[key] % (project.series, device)
-            bootstrap_files = ['ubuntu_zip', 'system_img', 'boot_img',
-                                     'recovery_img']
-            recovery_files = ['device_zip', 'ubuntu_zip']
-        else:
-            files['ubuntu_zip'] = None
-            files['device_zip'] = None
-            files['recovery_img'] = None
-            files['system_img'] = settings.device_file_img % (device,)
-            files['boot_img'] = settings.boot_file_img % (device,)
-            bootstrap_files = ['system_img', 'boot_img']
-            recovery_files = None
-        for key in files:
-            if not files[key]:
-                continue
-            files[key] = path.join(self._download_dir, files[key])
-        return files, bootstrap_files, recovery_files
-            
-
-    def store_hashes(self):
-        if not self._project.hash_content:
-            return None
-        with open(path.join(self._download_dir,
-                  self._project.hash_file_name), 'w') as f:
-            for line in self._project.hash_content:
-                f.write(line)
+def detect_device(serial, device=None):
+    '''If no argument passed determine them from the connected device.'''
+    # Check CyanogenMod property
+    if not device:
+        adb = AndroidBridge(serial)
+        adb.start()
+        device = adb.getprop('ro.cm.device').strip()
+    # Check Android property
+    if not device:
+        device = adb.getprop('ro.product.device').strip()
+    log.info('Device detected as %s' % device)
+    # property may not exist or we may not map it yet
+    if device not in settings.supported_devices:
+        raise EnvironmentError('Unsupported device, autodetect fails device')
+    return device
+
+
+#def setup_revision1(device, uri, download_dir, settings):
+#    if not uri:
+#        uri = settings.download_uri
+#    if not download_dir:
+#        build = get_ubuntu_stamp(uri)
+#        download_dir = get_full_path(
+#            os.path.join(settings.download_dir, build))
+#        setup_download_directory(download_dir)
+#    else:
+#        uri = None
+#    system = settings.device_file_img % device
+#    boot = settings.boot_file_img % (device,)
+#    hash_dict = {entry: load_hash(uri, '%s.md5sum' % entry, download_dir)
+#              for entry in (system, boot)}
+#    if uri:
+#        system_uri = '%s/%s' % (uri, system),
+#        boot_uri = '%s/%s' % (uri, boot),
+#    else:
+#        system_uri = None
+#        boot_uri = None
+#    system_file = resources.File(file_path=os.path.join(download_dir, system),
+#                                 file_uri=system_uri,
+#                                 file_hash=hashes[system],
+#                                 file_hash_func=hashlib.md5)
+#    boot_file = resources.File(file_path=os.path.join(download_dir, boot),
+#                               file_uri=boot_uri,
+#                               file_hash=hashes[boot],
+#                               file_hash_func=hashlib.md5)
+#    return projects.Android(boot=boot_file, system=system_file)
+
+
+def setup_cdimage_files(project_name, uri, download_dir, series,
+                        device, legacy=False):
+    downloads.setup_download_directory(download_dir)
+    templ_arch_any = settings.files_arch_any[project_name]
+    templ_arch_all = settings.files_arch_all[project_name]
+    file_names = {}
+    for key in templ_arch_any:
+        file_names[key] = templ_arch_any[key] % (series, device)
+    for key in templ_arch_all:
+        file_names[key] = templ_arch_all[key] % series
+    if legacy:
+        hash_func = hashlib.md5
+        hash_dict = {}
+        for key in file_names:
+            file_hash = hashes.load_hash(uri, '%s.md5sum' %
+                                         file_names[key], download_dir)
+            hash_dict[file_names[key]] = file_hash[file_names[key]]
+    else:
+        hash_func = hashlib.sha256
+        hash_dict = hashes.load_hash(uri, 'SHA256SUMS', download_dir)
+    files = {}
+    for key in file_names:
+        if uri:
+            file_uri = '%s/%s' % (uri, file_names[key])
+        else:
+            file_uri = None
+        files[key] = resources.File(
+            file_path=os.path.join(download_dir, file_names[key]),
+            file_uri=file_uri,
+            file_hash=hash_dict[file_names[key]],
+            file_hash_func=hash_func)
+    return files
+
+
+def setup_cdimage_touch(args):
+    device = detect_device(args.serial, args.device)
+    series = args.series
+    base_uri = '%s/%s' % (settings.cdimage_uri_base, args.project)
+
+    if args.base_path:
+        uri = None
+        download_dir = args.base_path
+
+    daily_uri = '%s/daily-preinstalled' % (base_uri, )
+    build = cdimage.get_build(daily_uri, args.pending)
+    uri = '%s/%s' % (daily_uri, build)
+    download_dir = downloads.get_full_path(
+        os.path.join(settings.download_dir, args.project, build))
+    files = setup_cdimage_files(
+        args.project, uri, download_dir, series, device)
+    return cdimage_project(files, args)
+
+
+def setup_cdimage_legacy(args):
+    series = args.series
+    uri = args.uri
+    device = detect_device(args.serial, args.device)
+    if args.base_path:
+        download_dir = args.base_path
+    elif args.revision or args.latest_revision:
+        build = args.build
+        uri = args.uri
+        download_dir = downloads.get_full_path(
+            os.path.join(args.project, series, build))
+    else:
+        base_uri = '%s/%s' % (settings.cdimage_uri_base, args.project)
+        daily_uri = '%s/daily-preinstalled' % (base_uri, )
+        build = cdimage.get_build(daily_uri)
+        uri = '%s/%s' % (daily_uri, build)
+        download_dir = downloads.get_full_path(
+            os.path.join(settings.download_dir, args.project, build))
+    files = setup_cdimage_files(
+        args.project, uri, download_dir, series, device, legacy=True)
+    return cdimage_project(files, args)
+
+
+def cdimage_project(files, args):
+    if args.bootstrap:
+        return projects.UbuntuTouchBootstrap(
+            system=files['system_img'],
+            boot=files['boot_img'],
+            recovery=files['recovery_img'],
+            ubuntu=files['ubuntu_zip'])
+    else:
+        if args.device_path:
+            files['device_zip'] = args.device_path
+        if args.ubuntu_path:
+            files['ubuntu_zip'] = args.ubuntu_path
+        return projects.UbuntuTouchRecovery(
+            device=files['device_zip'],
+            ubuntu=files['ubuntu_zip'],
+            wipe=args.wipe)
+
+
+def setup_ubuntu_system(args):
+    device = detect_device(args.serial, args.device)
+    if args.revision <= 0:
+        json = ubuntuimage.get_json_from_index(device, args.revision)
+    else:
+        raise EnvironmentError('Specific version retrieve not supported yet')
+    download_dir = downloads.get_full_path(os.path.join(
+        settings.download_dir, args.project, str(json['version'])))
+    uri = settings.system_image_uri
+    files, command_part = ubuntuimage.get_files(download_dir, uri, json)
+    recovery = cdimage.get_file(file_key='recovery_img',
+                                series=args.series,
+                                download_dir=download_dir,
+                                device=device)
+    return projects.UbuntuTouchSystem(
+        file_list=files,
+        command_part=command_part,
+        recovery=recovery)
+
+
+def setup_community(args):
+    config_dir = community.branch_project(args.device)
+    json_dict = community.load_manifest(config_dir)
+    download_dir = community.get_download_dir(
+        args.device, json_dict['revision'])
+    files = community.get_files(json_dict, download_dir, args.series)
+    return projects.UbuntuTouchRecovery(
+        device=files['device'],
+        ubuntu=files['ubuntu'],
+        storage=json_dict['storage'],
+        wipe=args.wipe)
+
+
+def list_revisions(args):
+    # Easy hack to get rid of the logger and inhibit requests from logging
+    log.setLevel(logging.FATAL)
+    revisions = cdimage.get_available_revisions(args.uri)
+    cdimage.display_revisions(revisions)

=== added file 'phabletutils/hashes.py'
--- phabletutils/hashes.py	1970-01-01 00:00:00 +0000
+++ phabletutils/hashes.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,66 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+import os.path
+
+from phabletutils import downloads
+
+log = logging.getLogger()
+
+
+def hash2dict(hash_file_content):
+    '''Returns a dictionary with the sha256 sums for all files.'''
+    if not hash_file_content:
+        log.debug('hash file is empty')
+        return None
+    hash_list = filter((lambda x: len(x) is not 0),
+                       hash_file_content.split('\n'))
+    hash_list = [h.split() for h in hash_list]
+    hash_dict = {}
+    for hash_entry in hash_list:
+        if hash_entry[1][0] == '*':
+            hash_entry[1] = hash_entry[1][1:]
+        hash_dict[hash_entry[1]] = hash_entry[0]
+    return hash_dict
+
+
+def load_hash(uri, artifact, download_dir=None):
+    if download_dir:
+        hash_path = os.path.join(download_dir, artifact)
+    else:
+        hash_path = None
+    hashes = {}
+    if hash_path and os.path.exists(hash_path):
+        with open(hash_path, 'r') as f:
+            file_hash_content = f.read()
+        hashes = hash2dict(file_hash_content)
+    if uri:
+        uri = '%s/%s' % (uri, artifact)
+        hash_content = downloads.get_content(uri)
+        if not hash_content:
+            raise RuntimeError('%s cannot be downloaded' % uri)
+        hashes.update(hash2dict(hash_content))
+        if hash_path:
+            log.debug('Storing hash to %s' % hash_path)
+            with downloads.flocked(hash_path):
+                with open(hash_path, 'w') as f:
+                    for key in hashes:
+                        f.write('%s %s\n' % (hashes[key], key))
+    if hashes:
+        return hashes
+    else:
+        raise RuntimeError('%s cannot be obtained for verifiaction' % artifact)

=== added file 'phabletutils/license.py'
--- phabletutils/license.py	1970-01-01 00:00:00 +0000
+++ phabletutils/license.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,51 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import logging
+
+log = logging.getLogger()
+
+
+def accepted(pathname):
+    '''
+    Remember that the user accepted the license.
+    '''
+    open(pathname, 'w').close()
+
+
+def has_accepted(pathname):
+    '''
+    Return True if the user accepted the license once.
+    '''
+    return os.path.exists(pathname)
+
+
+def query(message, accept_path):
+    '''Display end user agreement to continue with deployment.'''
+    try:
+        while True:
+            print message
+            print 'Do you accept? [yes|no]'
+            answer = raw_input().lower()
+            if answer == 'yes':
+                accepted(accept_path)
+                return True
+            elif answer == 'no':
+                return False
+    except KeyboardInterrupt:
+        log.error('Interruption detected, cancelling install')
+        return False

=== added file 'phabletutils/projects.py'
--- phabletutils/projects.py	1970-01-01 00:00:00 +0000
+++ phabletutils/projects.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,237 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Holds different projects or ways Ubuntu Touch is delivered."""
+
+import os
+import os.path
+import tempfile
+import logging
+import gzip
+
+from phabletutils.downloads import checksum_verify
+from phabletutils.resources import (File, SignedFile)
+from phabletutils import downloads
+from phabletutils import settings
+from time import sleep
+from textwrap import dedent
+
+log = logging.getLogger()
+
+
+def gunzip(file_path):
+    if not file_path.endswith('.gz'):
+        return file_path
+    target_path = file_path[:-3]
+    log.info('Decompressing %s' % file_path)
+    with open(target_path, 'wb') as target_file:
+        with gzip.open(file_path, 'r') as gzip_file:
+            for chunk in gzip_file.read():
+                target_file.write(chunk)
+    return target_path
+
+
+def wipe_device(adb):
+    log.info('Clearing /data and /cache')
+    adb.shell('mount /data')
+    adb.shell('rm -Rf /cache/* /data/* /data/.developer_mode')
+    adb.shell('mkdir /cache/recovery')
+    adb.shell('mkdir /data/media')
+
+
+class BaseProject(object):
+    """A provisioning mechanism for all the projects."""
+
+    def __init__(self, recovery=None, system=None, boot=None,
+                 device=None, ubuntu=None, wipe=False):
+        self._list = []
+        for item in (recovery, system, boot, device, ubuntu):
+            if item and not isinstance(item, File):
+                raise TypeError('%s is not of type File' % item)
+            elif item:
+                self._list.append(item)
+                if item.check and not item.uri and not item.verified:
+                    raise EnvironmentError(
+                        '%s is not on disk' % item.path)
+
+        self._recovery = recovery
+        self._system = system
+        self._boot = boot
+        self._device = device
+        self._ubuntu = ubuntu
+        self._wipe = wipe
+
+    def download(self):
+        """Downloads and verifies resources."""
+        download_list = filter((lambda x: x.check), self._list)
+        download_list = filter((lambda x: not x.verified), download_list)
+        log.debug('Download list %s' % download_list)
+        if not download_list:
+            log.info('Download not required')
+            return
+        for entry in download_list:
+            log.debug('Download entry %s %s' % (entry.path, entry.verified))
+            downloads.download(entry)
+            if entry.hash and \
+               not checksum_verify(entry.path, entry.hash, entry.hash_type):
+                raise EnvironmentError(
+                    'Checksum does not match after download for %s '
+                    'and hash %s' % (entry.path, entry.hash))
+        download_list = filter(lambda x: isinstance(x, SignedFile), self._list)
+        for entry in download_list:
+            downloads.download_sig(entry)
+
+    def install(self):
+        raise EnvironmentError('Requires implementation')
+
+
+class Android(BaseProject):
+    """Standard Android Project."""
+
+    def __init__(self, boot, system):
+        super(Android, self).__init__(boot=boot, system=system)
+
+    def install(self, adb, fastboot):
+        log.warning('Device needs to be unlocked for the following to work')
+        fastboot.flash('system', gunzip(self.system))
+        fastboot.flash('boot', self.boot)
+        log.info('Installation will complete soon and reboot into Android')
+        fastboot.reboot()
+
+
+class UbuntuTouchBootstrap(BaseProject):
+
+    def __init__(self, boot, system, recovery, ubuntu):
+        log.debug('UbuntuTouchBootstrap '
+                  'boot: %s, system: %s, recovery: %s, ubuntu: %s' %
+                  (boot.path, system.path, recovery.path, ubuntu.path))
+        super(UbuntuTouchBootstrap, self).__init__(
+            boot=boot, system=system, recovery=recovery, ubuntu=ubuntu,
+            wipe=True)
+
+    def install(self, adb, fastboot):
+        adb.reboot(bootloader=True)
+        log.warning('Device needs to be unlocked for the following to work')
+        fastboot.flash('system', self._system.path)
+        fastboot.flash('boot', self._boot.path)
+        fastboot.flash('recovery', self._recovery.path)
+        fastboot.boot(self._recovery.path)
+        sleep(15)
+        wipe_device(adb)
+        adb.push(self._ubuntu.path, '/sdcard/autodeploy.zip')
+        log.info('Deploying Ubuntu')
+        adb.reboot(recovery=True)
+        log.info('Installation will complete soon and reboot into Ubuntu')
+
+
+class UbuntuTouchRecovery(BaseProject):
+
+    recovery_script_template = dedent('''\
+                                      mount("{0}");
+                                      install_zip("{1}");
+                                      install_zip("{2}");
+                                      ''')
+
+    def __init__(self, device, ubuntu, storage='/sdcard/', wipe=False):
+        log.debug('UbuntuTouchRecovery device: %s, ubuntu: %s, wipe: %s' %
+                  (device.path, ubuntu.path, wipe))
+        super(UbuntuTouchRecovery, self).__init__(
+            ubuntu=ubuntu, device=device, wipe=wipe)
+        self._storage = storage
+
+    def install(self, adb, fastboot=None):
+        """
+        Deploys recovery files, recovery script and then reboots to install.
+        """
+        log.warning('The device needs to have a clockwork mod recovery image '
+                    '(or one that supports extendedcommands) '
+                    'in place for the provisioning to work')
+        adb.reboot(recovery=True)
+        sleep(20)
+        if self._wipe:
+            wipe_device(adb)
+        adb.shell('mount %s' % self._storage)
+        adb.push(self._device.path, self._storage)
+        adb.push(self._ubuntu.path, self._storage)
+        recovery_file = self.create_recovery_file()
+        adb.push(recovery_file, '/cache/recovery/extendedcommand')
+        adb.reboot(recovery=True)
+        log.info('Once completed the device should reboot into Ubuntu')
+        log.debug('Removing recovery file %s' % recovery_file)
+        os.unlink(recovery_file)
+
+    def create_recovery_file(self):
+        template = self.recovery_script_template
+        recovery_file = tempfile.NamedTemporaryFile(delete=False)
+        device = os.path.join(self._storage,
+                              os.path.basename(self._device.path))
+        ubuntu = os.path.join(self._storage,
+                              os.path.basename(self._ubuntu.path))
+        recovery_script = template.format(self._storage, device, ubuntu)
+        with recovery_file as output_file:
+            output_file.write(recovery_script)
+        return recovery_file.name
+
+
+class UbuntuTouchSystem(BaseProject):
+
+    ubuntu_recovery_script = dedent('''\
+        format data
+        format system
+        load_keyring image-master.tar.xz image-master.tar.xz.asc
+        load_keyring image-signing.tar.xz image-signing.tar.xz.asc
+        mount system
+        ''')
+
+    def __init__(self, file_list, recovery, command_part):
+        log.debug('UbuntuTouchSystem')
+        super(UbuntuTouchSystem, self).__init__(recovery=recovery, wipe=True)
+        for item in file_list:
+            if item and not isinstance(item, File):
+                raise TypeError('%s is not of type File' % item)
+            elif item:
+                self._list.append(item)
+        self._recovery_list = file_list
+        self._command_part = command_part
+
+    def install(self, adb, fastboot=None):
+        """
+        Deploys recovery files, recovery script and then reboots to install.
+        """
+        adb.reboot(recovery=True)
+        sleep(20)
+        wipe_device(adb)
+
+        for entry in self._recovery_list:
+            adb.push(entry.path, '/cache/recovery/')
+            try:
+                adb.push(entry.sig_path, '/cache/recovery/')
+            except AttributeError:
+                pass
+        adb.push(self.create_ubuntu_command_file(),
+                 '/cache/recovery/ubuntu_command')
+        adb.reboot(bootloader=True)
+        fastboot.flash('recovery', self._recovery.path)
+        fastboot.boot(self._recovery.path)
+        log.info('Once completed the device should reboot into Ubuntu')
+
+    def create_ubuntu_command_file(self):
+        ubuntu_command_file = tempfile.NamedTemporaryFile(delete=False)
+        with ubuntu_command_file as output_file:
+            output_file.write(settings.ubuntu_recovery_script)
+            output_file.write(self._command_part)
+            output_file.write('unmount system\n')
+        return ubuntu_command_file.name

=== added file 'phabletutils/resources.py'
--- phabletutils/resources.py	1970-01-01 00:00:00 +0000
+++ phabletutils/resources.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,82 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Resources for downloading and installing Ubuntu Touch."""
+
+import downloads
+import hashlib
+import logging
+
+log = logging.getLogger()
+
+
+class File(object):
+    """A file object that can be downloaded and flashed."""
+
+    @property
+    def uri(self):
+        return self._uri
+
+    @property
+    def path(self):
+        return self._path
+
+    @property
+    def verified(self):
+        return self._verified
+
+    @property
+    def hash(self):
+        return self._hash
+
+    @property
+    def hash_type(self):
+        return self._hash_func
+
+    @property
+    def check(self):
+        return self._check
+
+    def __init__(self, file_uri, file_path, check=True, file_hash=None,
+                 file_hash_func=hashlib.sha256):
+        self._uri = file_uri
+        self._path = file_path
+        self._hash = file_hash
+        self._hash_func = file_hash_func
+        self._check = check
+        if check and file_hash:
+            self._verified = downloads.checksum_verify(file_path, file_hash,
+                                                       file_hash_func)
+            log.debug('%s verified: %s' % (file_path, self._verified))
+        else:
+            self._verified = False
+
+
+class SignedFile(File):
+
+    def sig_uri(self):
+        return self._sig_uri
+
+    @property
+    def sig_path(self):
+        return self._sig_path
+
+    def __init__(self, file_uri, file_path, file_hash, sig_path, sig_uri,
+                 check=True, file_hash_func=hashlib.sha256):
+        super(SignedFile, self).__init__(
+            file_uri, file_path, check, file_hash, file_hash_func)
+        self._sig_path = sig_path
+        self.sig_uri = sig_uri

=== modified file 'phabletutils/settings.py'
--- phabletutils/settings.py	2013-07-20 02:14:56 +0000
+++ phabletutils/settings.py	2013-08-09 01:41:37 +0000
@@ -1,18 +1,18 @@
-#! /usr/bin/env python
-# This program is free software: you can redistribute it and/or modify it
-# under the terms of the the GNU General Public License version 3, as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful, but
-# WITHOUT ANY WARRANTY; without even the implied warranties of
-# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
-# PURPOSE.  See the applicable version of the GNU Lesser General Public
-# License for more details.
-#.
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-# Copyright (C) 2013 Canonical, Ltd.
 
 revision = 2
 default_series = 'saucy'
@@ -20,16 +20,14 @@
 system_image_uri = 'https://system-image.ubuntu.com'
 download_dir = 'phablet-flash'
 
-files = {
+files_arch_any = {
     'ubuntu-touch': {
-        'ubuntu_zip': '%s-preinstalled-touch-armhf.zip',
         'device_zip': '%s-preinstalled-touch-armel+%s.zip',
         'system_img': '%s-preinstalled-system-armel+%s.img',
         'boot_img': '%s-preinstalled-boot-armhf+%s.img',
         'recovery_img': '%s-preinstalled-recovery-armel+%s.img',
     },
     'ubuntu-touch-preview': {
-        'ubuntu_zip': '%s-preinstalled-phablet-armhf.zip',
         'device_zip': '%s-preinstalled-armel+%s.zip',
         'system_img': '%s-preinstalled-system-armel+%s.img',
         'boot_img': '%s-preinstalled-boot-armel+%s.img',
@@ -37,20 +35,21 @@
     },
 }
 
+files_arch_all = {
+    'ubuntu-touch': {
+        'ubuntu_zip': '%s-preinstalled-touch-armhf.zip',
+    },
+    'ubuntu-touch-preview': {
+        'ubuntu_zip': '%s-preinstalled-phablet-armhf.zip',
+    }
+}
+
 recovery_script_template = '''boot-recovery
---update_package={0}/{1}
---user_data_update_package={0}/{2}
+--update_package=/sdcard/{0}
+--user_data_update_package=/sdcard/{1}
 reboot
 '''
 
-ubuntu_recovery_script = '''format data
-format system
-load_keyring image-master.tar.xz image-master.tar.xz.asc
-load_keyring image-signing.tar.xz image-signing.tar.xz.asc
-mount system
-'''
-hash_file_name = 'SHA256SUMS'
-
 supported_devices = ('mako',
                      'maguro',
                      'manta',

=== modified file 'phabletutils/ubuntuimage.py'
--- phabletutils/ubuntuimage.py	2013-08-02 22:01:03 +0000
+++ phabletutils/ubuntuimage.py	2013-08-09 01:41:37 +0000
@@ -1,3 +1,4 @@
+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
 # Copyright (C) 2013 Canonical Ltd.
 # Author: Sergio Schvezov <[e-mail]>
 
@@ -15,50 +16,50 @@
 
 import json
 import os.path
-import requests
 
 from phabletutils import downloads
+from phabletutils import resources
 from phabletutils import settings
 
 
-def get_json_from_index(device, index=-1):
+def get_json_from_index(device, index):
     """Returns json index for device"""
+    index -= 1
     json_index_uri = '%s/daily/%s/index.json' % \
         (settings.system_image_uri, device)
-    json_index_request = requests.get(json_index_uri)
-    json_index = json.loads(json_index_request.content)
-    version_data = sorted([entry for entry in json_index['images']
-                          if entry['type'] == "full"],
-                          key=lambda entry: entry['version'])[index]
-    return version_data
-
-
-def download_images(download_dir, json_list):
+    json_content = downloads.get_content(json_index_uri)
+    if not json_content:
+        raise RuntimeError('%s cannot be retrieved' % json_index_uri)
+    json_index = json.loads(json_content)
+    json_dict = sorted([entry for entry in json_index['images']
+                       if entry['type'] == "full"],
+                       key=lambda entry: entry['version'])[index]
+    return json_dict
+
+
+def get_files(download_dir, uri, json):
     files = {}
-    files['updates'] = []
-    for entry in sorted(json_list['files'], key=lambda entry: entry['order']):
+    command_part = ''
+    files = []
+    for entry in sorted(json['files'], key=lambda entry: entry['order']):
         filename = entry['path'].split("/")[-1]
         signame = entry['signature'].split("/")[-1]
-        filename_path = os.path.join(download_dir, filename)
-        signame_path = os.path.join(download_dir, signame)
-        filename_uri = '%s/%s' % (settings.system_image_uri, entry['path'])
-        signame_uri = '%s/%s' % (settings.system_image_uri, entry['signature'])
-        with downloads.flocked(filename_path):
-            downloads.download(filename_uri, filename_path)
-            downloads.download(signame_uri, signame_path)
-        files['updates'].append({'filename': filename_path,
-                                 'signame': signame_path})
-    files['base'] = []
+        f = resources.SignedFile(
+            file_path=os.path.join(download_dir, filename),
+            sig_path=os.path.join(download_dir, signame),
+            file_uri='%s/%s' % (uri, entry['path']),
+            sig_uri='%s/%s' % (uri, entry['signature']),
+            file_hash=entry['checksum'])
+        files.append(f)
+        command_part += 'update %s %s\n' % (filename, signame)
     for keyring in ('image-master', 'image-signing'):
         filename = '%s.tar.xz' % keyring
         signame = '%s.asc' % filename
-        filename_path = os.path.join(download_dir, filename)
-        signame_path = os.path.join(download_dir, signame)
-        filename_uri = '%s/gpg/%s' % (settings.system_image_uri, filename)
-        signame_uri = '%s.asc' % filename_uri
-        with downloads.flocked(filename_path):
-            downloads.download(filename_uri, filename_path)
-            downloads.download(signame_uri, signame_path)
-        files['base'].append({'filename': filename_path,
-                              'signame': signame_path})
-    return files
+        f = resources.SignedFile(
+            file_path=os.path.join(download_dir, filename),
+            sig_path=os.path.join(download_dir, signame),
+            file_uri='%s/gpg/%s' % (uri, filename),
+            sig_uri='%s/gpg/%s.asc' % (uri, filename),
+            file_hash=None)
+        files.append(f)
+    return files, command_part

=== added file 'tests/index.json'
--- tests/index.json	1970-01-01 00:00:00 +0000
+++ tests/index.json	2013-08-09 01:41:37 +0000
@@ -0,0 +1,577 @@
+{
+    "global": {
+        "generated_at": "Wed Aug 07 11:30:01 UTC 2013"
+    },
+    "images": [
+        {
+            "description": "20130731.3",
+            "files": [
+                {
+                    "checksum": "7c9ce28bb751e25e80e5cee540ef1bdc929e97801528a14f066516f3ae2343d1",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130731.3.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130731.3.full.tar.xz.asc",
+                    "size": 264081876
+                },
+                {
+                    "checksum": "22ee6ded6902004e341a83c284218510a2b3a92c537a86140aa0daa76b7852f4",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130731.3.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130731.3.full.tar.xz.asc",
+                    "size": 37335764
+                },
+                {
+                    "checksum": "4181ff7e4f6f6262f5eb231e65735c277abf5a13993d68abb200ca9cd8136c4e",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130733.tar.xz",
+                    "signature": "/daily/mako/version-20130733.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "full",
+            "version": 20130733
+        },
+        {
+            "base": 20130732,
+            "description": "20130731.3",
+            "files": [
+                {
+                    "checksum": "002c10476be7a4db9d2fda3b9583eb67c2873b82c5f95483f6df7b253d7a8171",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130731.3.delta-20130731.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130731.3.delta-20130731.tar.xz.asc",
+                    "size": 14993132
+                },
+                {
+                    "checksum": "e557a50f29ecfe6265c405cbc35621f18e2b01225f9d9bfea01e30ddca752f43",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130731.3.delta-20130730.tar.xz",
+                    "signature": "/daily/mako/mako-20130731.3.delta-20130730.tar.xz.asc",
+                    "size": 34903260
+                },
+                {
+                    "checksum": "4181ff7e4f6f6262f5eb231e65735c277abf5a13993d68abb200ca9cd8136c4e",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130733.tar.xz",
+                    "signature": "/daily/mako/version-20130733.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "delta",
+            "version": 20130733
+        },
+        {
+            "description": "20130801",
+            "files": [
+                {
+                    "checksum": "963c3bf9367450ab17c47f670f91db1d15f0ef6bdcf194481b5f458379129d7e",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130801.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130801.full.tar.xz.asc",
+                    "size": 264000016
+                },
+                {
+                    "checksum": "eb2160d8e37b9e5af34a8cb8b9e6cc63105f759a953d928c896741feef7eed01",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130801.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130801.full.tar.xz.asc",
+                    "size": 37335924
+                },
+                {
+                    "checksum": "11da343b235e14a10a2537620e0dc397f4a23e8d5fc6f81e7fa86dcc9e80fa6a",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130800.tar.xz",
+                    "signature": "/daily/mako/version-20130800.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "full",
+            "version": 20130800
+        },
+        {
+            "base": 20130733,
+            "description": "20130801",
+            "files": [
+                {
+                    "checksum": "f81a98769b92a9ba2c61b75ee83aa7fa7417767788f518553ae195e0a846768f",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130801.delta-20130731.3.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130801.delta-20130731.3.tar.xz.asc",
+                    "size": 18625000
+                },
+                {
+                    "checksum": "e907a5a4740881080bb997e494df69e4b317ae938a2a689d0aa5cf7a086ea6a4",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130801.delta-20130731.3.tar.xz",
+                    "signature": "/daily/mako/mako-20130801.delta-20130731.3.tar.xz.asc",
+                    "size": 34903248
+                },
+                {
+                    "checksum": "11da343b235e14a10a2537620e0dc397f4a23e8d5fc6f81e7fa86dcc9e80fa6a",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130800.tar.xz",
+                    "signature": "/daily/mako/version-20130800.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "delta",
+            "version": 20130800
+        },
+        {
+            "description": "20130802.1",
+            "files": [
+                {
+                    "checksum": "05969428a0502d021ee91945f28bfd9884f774f406fe7e0578b09e26f521b171",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130802.1.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130802.1.full.tar.xz.asc",
+                    "size": 264078540
+                },
+                {
+                    "checksum": "269a33c87c6a995f068293776aa10f4a74d0cbb8101a236e0fedb375aff1df63",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130802.1.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130802.1.full.tar.xz.asc",
+                    "size": 37342212
+                },
+                {
+                    "checksum": "b2f4f1b04add4d47209598acb78137e3eb31ef1997aa92f67b17fcba8ea2911c",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130801.tar.xz",
+                    "signature": "/daily/mako/version-20130801.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "full",
+            "version": 20130801
+        },
+        {
+            "base": 20130800,
+            "description": "20130802.1",
+            "files": [
+                {
+                    "checksum": "d3c80a6144159933d037a605052b1c56ad4edae05ba31c73841c4bc80df59f6e",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130802.1.delta-20130801.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130802.1.delta-20130801.tar.xz.asc",
+                    "size": 18381112
+                },
+                {
+                    "checksum": "0fe99c8d57cda68dc757bb10d039f26895c822d7e5d6efb96f9456efd65c384f",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130802.1.delta-20130801.tar.xz",
+                    "signature": "/daily/mako/mako-20130802.1.delta-20130801.tar.xz.asc",
+                    "size": 37098756
+                },
+                {
+                    "checksum": "b2f4f1b04add4d47209598acb78137e3eb31ef1997aa92f67b17fcba8ea2911c",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130801.tar.xz",
+                    "signature": "/daily/mako/version-20130801.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "delta",
+            "version": 20130801
+        },
+        {
+            "description": "20130802.2",
+            "files": [
+                {
+                    "checksum": "253139a135389d920ce446d258e4b0c779c51780c1ad747641a6cb11a3331634",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130802.2.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130802.2.full.tar.xz.asc",
+                    "size": 264050500
+                },
+                {
+                    "checksum": "490fc1586ea46d98719f2d53ae3044c31ef065d7d353071aed9ca4af43e37ceb",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130802.2.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130802.2.full.tar.xz.asc",
+                    "size": 37335972
+                },
+                {
+                    "checksum": "64fbb613532c31d2ee0d6141a2cba656c3247f39a257b8eed67fa1d18f481983",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130802.tar.xz",
+                    "signature": "/daily/mako/version-20130802.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "full",
+            "version": 20130802
+        },
+        {
+            "base": 20130801,
+            "description": "20130802.2",
+            "files": [
+                {
+                    "checksum": "193fde8008c63009397cb5d470c0d81c3547583ea5a28e1c1dc0c7be57ab59cc",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130802.2.delta-20130802.1.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130802.2.delta-20130802.1.tar.xz.asc",
+                    "size": 19171116
+                },
+                {
+                    "checksum": "8f3747431d40a45e52c780116f92939ac4cd071d63a80d0c9031b1da6b6241ac",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130802.2.delta-20130802.1.tar.xz",
+                    "signature": "/daily/mako/mako-20130802.2.delta-20130802.1.tar.xz.asc",
+                    "size": 34996120
+                },
+                {
+                    "checksum": "64fbb613532c31d2ee0d6141a2cba656c3247f39a257b8eed67fa1d18f481983",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130802.tar.xz",
+                    "signature": "/daily/mako/version-20130802.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "delta",
+            "version": 20130802
+        },
+        {
+            "description": "20130803",
+            "files": [
+                {
+                    "checksum": "c0bc116b31adcb88b4587f8db0af9dcc6c6a2be839ea6013c5bd970bf9625467",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130803.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130803.full.tar.xz.asc",
+                    "size": 264111912
+                },
+                {
+                    "checksum": "02730cefdb71d35ce971f9494b0fedb775e1fd08cb65a8826fd74b3ebdba55d0",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130803.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130803.full.tar.xz.asc",
+                    "size": 37344168
+                },
+                {
+                    "checksum": "3b4f2fdce15f5f3ececdc73b2b58f512717da37abadd50d9ac08fff4dcafad00",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130803.tar.xz",
+                    "signature": "/daily/mako/version-20130803.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "full",
+            "version": 20130803
+        },
+        {
+            "base": 20130802,
+            "description": "20130803",
+            "files": [
+                {
+                    "checksum": "fd11b3944464bde87f7d187979eef556ce33a3f5f6fff00808bffeaea906b0ee",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130803.delta-20130802.2.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130803.delta-20130802.2.tar.xz.asc",
+                    "size": 14519416
+                },
+                {
+                    "checksum": "60f988ece3a4a092e0f05aa326feb655d67b68e2a53736bd4aa6717a3e0d26db",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130803.delta-20130802.2.tar.xz",
+                    "signature": "/daily/mako/mako-20130803.delta-20130802.2.tar.xz.asc",
+                    "size": 34918944
+                },
+                {
+                    "checksum": "3b4f2fdce15f5f3ececdc73b2b58f512717da37abadd50d9ac08fff4dcafad00",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130803.tar.xz",
+                    "signature": "/daily/mako/version-20130803.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "delta",
+            "version": 20130803
+        },
+        {
+            "description": "20130804",
+            "files": [
+                {
+                    "checksum": "f3c1273666c71cb3fd74d9762783c016794cf36f5aab3fa01329460966293c6c",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130804.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130804.full.tar.xz.asc",
+                    "size": 264089004
+                },
+                {
+                    "checksum": "5f49821d302631c5de2f5df838b168b2c8290c0a9a4a510533464d92e4b9b638",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130804.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130804.full.tar.xz.asc",
+                    "size": 37335780
+                },
+                {
+                    "checksum": "e2619bd73b47539e23ebfbc29620baf8a6d56c3cf850141e2f9b3c8973e00342",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130804.tar.xz",
+                    "signature": "/daily/mako/version-20130804.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "full",
+            "version": 20130804
+        },
+        {
+            "base": 20130803,
+            "description": "20130804",
+            "files": [
+                {
+                    "checksum": "80c02eb1269adfd3b641593a2a6c93a4f4c570fbce1a90020073295276a1c0ba",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130804.delta-20130803.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130804.delta-20130803.tar.xz.asc",
+                    "size": 16620436
+                },
+                {
+                    "checksum": "99d0eb9f2315675a427c899ed308ab2ce11694675c979a305800f1949e008083",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130804.delta-20130803.tar.xz",
+                    "signature": "/daily/mako/mako-20130804.delta-20130803.tar.xz.asc",
+                    "size": 34913316
+                },
+                {
+                    "checksum": "e2619bd73b47539e23ebfbc29620baf8a6d56c3cf850141e2f9b3c8973e00342",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130804.tar.xz",
+                    "signature": "/daily/mako/version-20130804.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "delta",
+            "version": 20130804
+        },
+        {
+            "description": "20130805",
+            "files": [
+                {
+                    "checksum": "6862856f1e029fd86883e1b28cc4aa959190ff3fdcaeb3c4115184c2942d21ee",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130805.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130805.full.tar.xz.asc",
+                    "size": 264023912
+                },
+                {
+                    "checksum": "c9f9b8644d5573cd73727a9a506ca46663cf22b6bae82d0dad90f3e53f43079c",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130805.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130805.full.tar.xz.asc",
+                    "size": 37343256
+                },
+                {
+                    "checksum": "915f703e637a509c9947890983427ec6d44fa53ee5a45e866acbd76c5c7d2fcb",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130805.tar.xz",
+                    "signature": "/daily/mako/version-20130805.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "full",
+            "version": 20130805
+        },
+        {
+            "base": 20130804,
+            "description": "20130805",
+            "files": [
+                {
+                    "checksum": "8d65c5deb8723db8df7e918790f98bfe4304490d7bde96af85fba0989ff04fe8",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130805.delta-20130804.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130805.delta-20130804.tar.xz.asc",
+                    "size": 15166336
+                },
+                {
+                    "checksum": "401400168374635c26c320218518565d567bd3c38d0a4ffc2ff750207702d070",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130805.delta-20130804.tar.xz",
+                    "signature": "/daily/mako/mako-20130805.delta-20130804.tar.xz.asc",
+                    "size": 34921568
+                },
+                {
+                    "checksum": "915f703e637a509c9947890983427ec6d44fa53ee5a45e866acbd76c5c7d2fcb",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130805.tar.xz",
+                    "signature": "/daily/mako/version-20130805.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "delta",
+            "version": 20130805
+        },
+        {
+            "description": "20130806",
+            "files": [
+                {
+                    "checksum": "cd06ae29bc3d57def4d216be24d857a0c01367f65626651459a9478c42a8d80c",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130806.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130806.full.tar.xz.asc",
+                    "size": 264142320
+                },
+                {
+                    "checksum": "db5a43ee320eac7474a98beb0bbcc43648ff17f90adf581d19200bfb5bda0dce",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130806.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130806.full.tar.xz.asc",
+                    "size": 42475196
+                },
+                {
+                    "checksum": "3a17c0ae8f9a42c26aee06313ebc619f0b2a24385f7cbf3df519698ab9078acf",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130806.tar.xz",
+                    "signature": "/daily/mako/version-20130806.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "full",
+            "version": 20130806
+        },
+        {
+            "base": 20130805,
+            "description": "20130806",
+            "files": [
+                {
+                    "checksum": "0acc8ef06a657d74117a759b7e0fff0ba7a69e332aec15f8dbc7d1716d04fbbb",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130806.delta-20130805.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130806.delta-20130805.tar.xz.asc",
+                    "size": 17806408
+                },
+                {
+                    "checksum": "59d1604655efc0cfc582b97bae1111344f273b49c3df259647fe6fb063448ebb",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130806.delta-20130805.tar.xz",
+                    "signature": "/daily/mako/mako-20130806.delta-20130805.tar.xz.asc",
+                    "size": 42221272
+                },
+                {
+                    "checksum": "3a17c0ae8f9a42c26aee06313ebc619f0b2a24385f7cbf3df519698ab9078acf",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130806.tar.xz",
+                    "signature": "/daily/mako/version-20130806.tar.xz.asc",
+                    "size": 196
+                }
+            ],
+            "type": "delta",
+            "version": 20130806
+        },
+        {
+            "description": "20130806.1",
+            "files": [
+                {
+                    "checksum": "b42156ae0b171c3425cb832d46cc4a86302a943beea18a29132ba3f84d978a02",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130806.1.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130806.1.full.tar.xz.asc",
+                    "size": 264114544
+                },
+                {
+                    "checksum": "6b5f8a78e94554ad59a3bb7f81550cce446d17d25d455759a77c6435d70de715",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130806.1.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130806.1.full.tar.xz.asc",
+                    "size": 36869564
+                },
+                {
+                    "checksum": "aa1fbad7150d4c0a2cc23e78672678aaeeaa26ac4e57a65fb635bf62c898955c",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130807.tar.xz",
+                    "signature": "/daily/mako/version-20130807.tar.xz.asc",
+                    "size": 188
+                }
+            ],
+            "type": "full",
+            "version": 20130807
+        },
+        {
+            "base": 20130806,
+            "description": "20130806.1",
+            "files": [
+                {
+                    "checksum": "3d6283525d795a34f98e2b996ae5bf7ce70892ca79026c776fca973b1080b30e",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130806.1.delta-20130806.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130806.1.delta-20130806.tar.xz.asc",
+                    "size": 14764572
+                },
+                {
+                    "checksum": "3e02f4a2ae19f47ba1bd10c2f236b46b368d012ede8003cb11fa3afbe0d7683f",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130806.1.delta-20130806.tar.xz",
+                    "signature": "/daily/mako/mako-20130806.1.delta-20130806.tar.xz.asc",
+                    "size": 34428224
+                },
+                {
+                    "checksum": "aa1fbad7150d4c0a2cc23e78672678aaeeaa26ac4e57a65fb635bf62c898955c",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130807.tar.xz",
+                    "signature": "/daily/mako/version-20130807.tar.xz.asc",
+                    "size": 188
+                }
+            ],
+            "type": "delta",
+            "version": 20130807
+        },
+        {
+            "description": "20130807",
+            "files": [
+                {
+                    "checksum": "5a615f41c92c562496e650c2ee5831a66da02c5435c5270d6b594aec6e30511d",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130807.full.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130807.full.tar.xz.asc",
+                    "size": 264539288
+                },
+                {
+                    "checksum": "1a39b78b220660fe0c7fd8c30febe92009ad5306610e41f7cdf838da8290c59a",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130807.full.tar.xz",
+                    "signature": "/daily/mako/mako-20130807.full.tar.xz.asc",
+                    "size": 36875496
+                },
+                {
+                    "checksum": "287eb50401487dc217f6a614fa03ae1bfb7fbead48f196692cf090667c98353f",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130808.tar.xz",
+                    "signature": "/daily/mako/version-20130808.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "full",
+            "version": 20130808
+        },
+        {
+            "base": 20130807,
+            "description": "20130807",
+            "files": [
+                {
+                    "checksum": "1e48af06cb25e899a536301c36f212f06c0479be69785d1569d2795099485ab5",
+                    "order": 0,
+                    "path": "/daily/ubuntu/ubuntu-20130807.delta-20130806.1.tar.xz",
+                    "signature": "/daily/ubuntu/ubuntu-20130807.delta-20130806.1.tar.xz.asc",
+                    "size": 17889552
+                },
+                {
+                    "checksum": "10000fae3aa0a567bb10cc5efc6552bb01ee6ec96a689f0adf886ad3e202b0c7",
+                    "order": 1,
+                    "path": "/daily/mako/mako-20130807.delta-20130806.1.tar.xz",
+                    "signature": "/daily/mako/mako-20130807.delta-20130806.1.tar.xz.asc",
+                    "size": 34435132
+                },
+                {
+                    "checksum": "287eb50401487dc217f6a614fa03ae1bfb7fbead48f196692cf090667c98353f",
+                    "order": 2,
+                    "path": "/daily/mako/version-20130808.tar.xz",
+                    "signature": "/daily/mako/version-20130808.tar.xz.asc",
+                    "size": 192
+                }
+            ],
+            "type": "delta",
+            "version": 20130808
+        }
+    ]
+}

=== modified file 'tests/test_cdimage.py'
--- tests/test_cdimage.py	2013-07-14 20:40:06 +0000
+++ tests/test_cdimage.py	2013-08-09 01:41:37 +0000
@@ -37,7 +37,7 @@
         process_mock.return_value = target_build
         self.rsync_call.append('%s/current' % self.rsync_uri_part)
         # when
-        build = cdimage.get_latest_current(self.cdimage_uri)
+        build = cdimage.get_build(self.cdimage_uri)
         # then
         self.assertThat(build, Equals(target_build))
         process_mock.assert_called_once_with(self.rsync_call)
@@ -49,7 +49,7 @@
         process_mock.return_value = target_build
         self.rsync_call.append('%s/pending' % self.rsync_uri_part)
         # when
-        build = cdimage.get_latest_pending(self.cdimage_uri)
+        build = cdimage.get_build(self.cdimage_uri, pending=True)
         # then
         self.assertThat(build, Equals(target_build))
-        process_mock.assert_called_once_with(self.rsync_call)
\ No newline at end of file
+        process_mock.assert_called_once_with(self.rsync_call)

=== renamed file 'tests/test_environment.py' => 'tests/test_community.py'
--- tests/test_environment.py	2013-07-14 20:40:06 +0000
+++ tests/test_community.py	2013-08-09 01:41:37 +0000
@@ -15,134 +15,115 @@
 
 """Unit tests for phabletutils.environment."""
 
-from mock import patch
+import json
+import shutil
+import tempfile
+
 from os import path
-from phabletutils import environment
-from phabletutils import settings
+from phabletutils import community
 from testtools import TestCase
 from testtools.matchers import Equals
-from testscenarios import TestWithScenarios
-from testscenarios import multiply_scenarios
-
-
-class FakeSettings(object):
-    download_dir = 'fake_download_dir'
-    device_file_img = 'device-%s.img'
-    boot_file_img = 'boot-%s.img'
-
-
-@patch('phabletutils.environment.get_ubuntu_stamp')
-class TestPlainEnvironments(TestCase):
-    def setUp(self):
-        super(TestPlainEnvironments, self).setUp()
-        self.device = 'mako'
+from testtools.matchers import Is
+
+
+class TestCommunityManifestLoad(TestCase):
+
+    def setUp(self):
+        super(TestCommunityManifestLoad, self).setUp()
+        self.device_dir = tempfile.mkdtemp()
+
+    def tearDown(self):
+        super(TestCommunityManifestLoad, self).tearDown()
+        shutil.rmtree(self.device_dir)
+
+    def testSimpleDeviceManifest(self):
+        # given
+        manifest = {'device': 'http://somelocation.com/device.zip'}
+        with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
+                f.write(json.dumps(manifest))
+        # when
+        loaded_manifest = community.load_manifest(self.device_dir)
+        # then
+        self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
+        self.assertThat(loaded_manifest['ubuntu'], Is(None))
+        self.assertThat(loaded_manifest['revision'], Is(None))
+        self.assertThat(loaded_manifest['storage'], Equals('/sdcard/'))
+
+    def testSimpleDeviceAndUbuntuManifest(self):
+        # given
+        manifest = {'device': 'http://somelocation.com/device.zip',
+                    'ubuntu': 'http://somelocation.com/ubuntu.zip'}
+        with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
+                f.write(json.dumps(manifest))
+        # when
+        loaded_manifest = community.load_manifest(self.device_dir)
+        # then
+        self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
+        self.assertThat(loaded_manifest['ubuntu'], Equals(manifest['ubuntu']))
+        self.assertThat(loaded_manifest['revision'], Is(None))
+        self.assertThat(loaded_manifest['storage'], Equals('/sdcard/'))
+
+    def testSimpleDeviceAndUbuntuRevisionManifest(self):
+        # given
+        manifest = {'device': 'http://somelocation.com/device.zip',
+                    'ubuntu': 'http://somelocation.com/ubuntu.zip',
+                    'revision': 'unstable'}
+        with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
+                f.write(json.dumps(manifest))
+        # when
+        loaded_manifest = community.load_manifest(self.device_dir)
+        # then
+        self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
+        self.assertThat(loaded_manifest['ubuntu'], Equals(manifest['ubuntu']))
+        self.assertThat(loaded_manifest['revision'],
+                        Equals(manifest['revision']))
+        self.assertThat(loaded_manifest['storage'], Equals('/sdcard/'))
+
+    def testSimpleDeviceAndUbuntuRevisionStorageManifest(self):
+        # given
+        manifest = {'device': 'http://somelocation.com/device.zip',
+                    'ubuntu': 'http://somelocation.com/ubuntu.zip',
+                    'storage': '/emmc/',
+                    'revision': 'unstable'}
+        with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
+                f.write(json.dumps(manifest))
+        # when
+        loaded_manifest = community.load_manifest(self.device_dir)
+        # then
+        self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
+        self.assertThat(loaded_manifest['ubuntu'], Equals(manifest['ubuntu']))
+        self.assertThat(loaded_manifest['revision'],
+                        Equals(manifest['revision']))
+        self.assertThat(loaded_manifest['storage'],
+                        Equals(manifest['storage']))
+
+
+class TestCommunityFiles(TestCase):
+
+    def setUp(self):
+        super(TestCommunityFiles, self).setUp()
+        self.download_dir = '/downloads'
         self.series = 'saucy'
-        self.settings = FakeSettings()
-        self.uri = 'http://download.com/artifacts'
-        self.build = '10'
-
-    def testNonCdimage(self, get_ubuntu_stamp):
-        # given
-        get_ubuntu_stamp.return_value = self.build
-        download_dir = path.join(self.settings.download_dir, self.build)
-        full_download_dir = environment.get_download_dir_full_path(
-            download_dir)
-        system_img_path = path.join(full_download_dir,
-            self.settings.device_file_img % self.device)
-        boot_img_path = path.join(full_download_dir,
-                                  self.settings.boot_file_img % self.device)
-        # when
-        env = environment.Environment(self.uri, self.device, self.series,
-                                      None, None, False, None, None,
-                                      self.settings)
-        # then
-        self.assertFalse(env.project)
-        self.assertThat(env.download_dir, Equals(full_download_dir))
-        self.assertThat(env.download_uri, Equals(self.uri))
-        self.assertThat(env.system_img_path, Equals(system_img_path))
-        self.assertThat(env.boot_img_path, Equals(boot_img_path))
-        self.assertFalse(env.recovery_img_path)
-
-
-def get_file_paths(files, download_dir, device, series):
-    if not series:
-        series = settings.default_series
-    replacements = (series, device)
-    files = {
-        'system_img': path.join(download_dir,
-                                files['system_img'] % replacements),
-        'boot_img': path.join(download_dir,
-                              files['boot_img'] % replacements),
-        'recovery_img': path.join(download_dir,
-                                  files['recovery_img'] % replacements),
-        'device_zip': path.join(download_dir,
-                                files['device_zip'] % replacements),
-        'ubuntu_zip': path.join(download_dir, files['ubuntu_zip'] % series),
-    }
-    return files
-
-
-@patch('phabletutils.cdimage.get_sha256_content')
-@patch('phabletutils.cdimage.get_latest_current')
-class TestCdimageEnvironments(TestWithScenarios, TestCase):
-
-    scenarios = multiply_scenarios(
-                [('legacy touch', dict(legacy=True)),
-                 ('touch', dict(legacy=False))],
-                [('use default series', dict(series=None)),
-                 ('set series', dict(series='raring'))],
-                [('base path', dict(base_path='/prev_download')),
-                 ('no base path', dict(base_path=None))])
-
-    def setUp(self):
-        super(TestCdimageEnvironments, self).setUp()
-        if self.legacy:
-            self.project = 'ubuntu-touch-preview'
-        else:
-            self.project = 'ubuntu-touch'
-        self.device = 'manta'
-        self.build = '20130606'
-        self.download_dir = path.join(settings.download_dir,
-                                      self.project, self.build)
-        if not self.base_path:
-            self.full_download_dir = environment.get_download_dir_full_path(
-                    self.download_dir)
-            self.download_uri = path.join(settings.cdimage_uri_base,
-                    self.project, 'daily-preinstalled', self.build)
-        else:
-            self.full_download_dir = self.base_path
-            self.download_uri = None
-        self.files = get_file_paths(settings.files[self.project],
-                                    self.full_download_dir,
-                                    self.device, self.series)
-
-    def testEnvironmentSetup(self, cdimage_get_latest, cdimage_sha256):
-        # given
-        cdimage_get_latest.return_value = self.build
-        cdimage_sha256.return_value = 'sha'
-        # when
-        env = environment.Environment(None, self.device, self.series,
-                                      None, None, False, self.legacy,
-                                      self.base_path, settings)
-        # then
-        self.assertTrue(env.project)
-        self.assertThat(env.download_dir, Equals(self.full_download_dir))
-        self.assertThat(env.download_uri, Equals(self.download_uri))
-        self.assertThat(env.system_img_path, Equals(self.files['system_img']))
-        self.assertThat(env.boot_img_path, Equals(self.files['boot_img']))
-        self.assertThat(env.device_zip_path, Equals(self.files['device_zip']))
-        self.assertThat(env.ubuntu_zip_path, Equals(self.files['ubuntu_zip']))
-        self.assertThat(env.recovery_img_path,
-                Equals(self.files['recovery_img']))
-        # Check recovery file list
-        self.assertTrue(self.files['device_zip'] in env.recovery_files)
-        self.assertTrue(self.files['ubuntu_zip'] in env.recovery_files)
-        self.assertFalse(self.files['recovery_img'] in env.recovery_files)
-        self.assertFalse(self.files['system_img'] in env.recovery_files)
-        self.assertFalse(self.files['boot_img'] in env.recovery_files)
-        # Check bootstrap file list
-        self.assertFalse(self.files['device_zip'] in env.bootstrap_files)
-        self.assertTrue(self.files['ubuntu_zip'] in env.bootstrap_files)
-        self.assertTrue(self.files['recovery_img'] in env.bootstrap_files)
-        self.assertTrue(self.files['system_img'] in env.bootstrap_files)
-        self.assertTrue(self.files['boot_img'] in env.bootstrap_files)
\ No newline at end of file
+
+    def tearDown(self):
+        super(TestCommunityFiles, self).tearDown()
+
+    def testSimpleDeviceAndUbuntu(self):
+        # given
+        manifest = {'device': 'http://somelocation.com/device.zip',
+                    'ubuntu': 'http://somelocation.com/ubuntu.zip'}
+        # when
+        files = community.get_files(manifest, self.download_dir, self.series)
+        #then
+        self.assertThat(files['device'].uri, Equals(manifest['device']))
+        self.assertThat(files['device'].hash, Is(None))
+        self.assertThat(files['device'].verified, Is(False))
+        self.assertThat(files['device'].path,
+                        Equals(path.join(self.download_dir,
+                               path.basename(manifest['device']))))
+        self.assertThat(files['ubuntu'].uri, Equals(manifest['ubuntu']))
+        self.assertThat(files['ubuntu'].hash, Is(None))
+        self.assertThat(files['ubuntu'].verified, Is(False))
+        self.assertThat(files['ubuntu'].path,
+                        Equals(path.join(self.download_dir,
+                               path.basename(manifest['ubuntu']))))

=== added file 'tests/test_ubuntuimage.py'
--- tests/test_ubuntuimage.py	1970-01-01 00:00:00 +0000
+++ tests/test_ubuntuimage.py	2013-08-09 01:41:37 +0000
@@ -0,0 +1,54 @@
+# Copyright (C) 2013 Canonical Ltd.
+# Author: Sergio Schvezov <[e-mail]>
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""Unit tests for phabletutils.environment."""
+
+from mock import patch
+from phabletutils import ubuntuimage
+from testtools import TestCase
+from testtools.matchers import Equals
+from testtools.matchers import HasLength
+
+
+@patch('phabletutils.downloads.get_content')
+class TestUbuntuImage(TestCase):
+
+    def setUp(self):
+        super(TestUbuntuImage, self).setUp()
+        self.device = 'mako'
+        with open('tests/index.json', 'r') as f:
+            self.json_content = f.read()
+
+    def testGetLatestIndex(self, get_content_mock):
+        # given
+        get_content_mock.return_value = self.json_content
+        # when
+        json_dict = ubuntuimage.get_json_from_index(self.device, 0)
+        # then
+        self.assertThat(json_dict['version'], Equals(20130808))
+        self.assertThat(json_dict['description'], Equals('20130807'))
+        self.assertThat(json_dict['type'], Equals('full'))
+        self.assertThat(json_dict['files'], HasLength(3))
+
+    def testGetPreviousIndex(self, get_content_mock):
+        # given
+        get_content_mock.return_value = self.json_content
+        # when
+        json_dict = ubuntuimage.get_json_from_index(self.device, -1)
+        # then
+        self.assertThat(json_dict['version'], Equals(20130807))
+        self.assertThat(json_dict['description'], Equals('20130806.1'))
+        self.assertThat(json_dict['type'], Equals('full'))
+        self.assertThat(json_dict['files'], HasLength(3))