Merge lp:~sergiusens/phablet-tools/flash_change into lp:phablet-tools

Proposed by Sergio Schvezov on 2013-07-31
Status: Merged
Approved by: Ricardo Salveti on 2013-08-09
Approved revision: 191
Merged at revision: 150
Proposed branch: lp:~sergiusens/phablet-tools/flash_change
Merge into: lp:phablet-tools
Diff against target: 3365 lines (+2046/-1044) 20 files modified
To merge this branch: bzr merge lp:~sergiusens/phablet-tools/flash_change
Reviewer Review Type Date Requested Status
PS Jenkins bot continuous-integration Approve on 2013-08-09
Ricardo Salveti 2013-08-01 Approve on 2013-08-09
Paul Larson (community) 2013-08-01 Approve on 2013-08-08
Daniel Holbach 2013-08-01 Approve on 2013-08-08
Andy Doan (community) 2013-08-01 Approve on 2013-08-07
Omer Akram 2013-08-07 Pending
Alan Pope ㋛ 2013-08-01 Pending
Ubuntu Phablet Team 2013-07-31 Pending
Review via email: mp+177927@code.launchpad.net

Commit Message

Refactoring phablet-flash to accommodate multiple flashing scenarios. Adding community flashing support.

Description of the Change

Don't worry about the conflicts for now. Just branch straight and check the code if you are reviewing code.

Major change:
positional params: community, cdimage-touch, cdimage-legacy, ubuntu-system

To flash and ubuntu image based upgrade:
phablet-flash ubuntu-system

To flash cdimage touch
phablet-flash cdimage-touch (--pending to get pending)

cdimage legacy:
phablet-flash cdimage-legacy (--list-revisions, --latest and --revision work as usual, just not exposed to other params)

community:
phablet-flash community --device i9100 (will download, not sure if flash as I don't have the device).

I'm not using /cache/recovery/command anymore, instead /cache/recovery/extendedcomands to support community deploys.

Feel free to test and report early issues. I'm still shuffling some code around. You might notice some duplication, that's on purpose, things that might look similar today may change tomorrow and I've had a hard time with this already.

Still missing: adhoc positional argument and device and ubuntu override (I guess img too, but we aren't distributing those).

To post a comment you must log in.
Daniel Holbach (dholbach) wrote :

daniel@daydream:~/flash_change$ ./phablet-flash community --device i9100
Traceback (most recent call last):
  File "./phablet-flash", line 24, in <module>
    from phabletutils import arguments
  File "/home/daniel/flash_change/phabletutils/arguments.py", line 21, in <module>
    from phabletutils import environment
  File "/home/daniel/flash_change/phabletutils/environment.py", line 26, in <module>
    from phabletutils import community
ImportError: cannot import name community
daniel@daydream:~/flash_change$

Could it be you forgot to "bzr add community*" somewhere?

Daniel Holbach (dholbach) wrote :

A few small things:
phablet-flash:17: 'argparse' imported but unused
phablet-flash:20: 'subprocess' imported but unused
phablet-flash:22: 'sleep' imported but unused
phablet-flash:25: 'environment' imported but unused
phablet-flash:27: 'ubuntuimage' imported but unused

Apart from that the code looks cleaner now. There were some things I was a bit unsure about, like when adb is run, and with which arguments, but it seems like a lot of the code was just moved somewhere else.

Good work and thanks for fixing bug 1201811.

review: Approve
Sergio Schvezov (sergiusens) wrote :

> A few small things:
> phablet-flash:17: 'argparse' imported but unused
> phablet-flash:20: 'subprocess' imported but unused
> phablet-flash:22: 'sleep' imported but unused
> phablet-flash:25: 'environment' imported but unused
> phablet-flash:27: 'ubuntuimage' imported but unused

fixed in 169

> Apart from that the code looks cleaner now. There were some things I was a bit
> unsure about, like when adb is run, and with which arguments, but it seems
> like a lot of the code was just moved somewhere else.

thanks, it's easy when you can break command line backwards compatibility :-)

> Good work and thanks for fixing bug 1201811.

Was fun :-D

Andy Doan (doanac) wrote :
Sergio Schvezov (sergiusens) wrote :

> looks like the flock logic from http://bazaar.launchpad.net/~phablet-team
> /phablet-tools/trunk/revision/148 is being lost?

oh, I didn't apply it yet, but it will be much easier here :-)

Daniel Holbach (dholbach) wrote :

Does anyone else want to comment?

Sergio Schvezov (sergiusens) wrote :

> looks like the flock logic from http://bazaar.launchpad.net/~phablet-team
> /phablet-tools/trunk/revision/148 is being lost?

It's here now
http://bazaar.launchpad.net/~sergiusens/phablet-tools/flash_change/revision/176

Andy Doan (doanac) wrote :

revno 176 adds the flock back. due to the way your code is now structured you could probably rename the the "flocked" function to be "_flocked" since its only used in that module. not a big deal though.

I tested the code though and it still works for me. We may want to send an email out just before this lands in the archive so that people are aware. For instance, its going to require us to update our provision script in automation.

review: Approve
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Continuous integration, rev:179
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~sergiusens/phablet-tools/flash_change/+merge/177927/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/125/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/68
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/68
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/68

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/125/rebuild

review: Needs Fixing (continuous-integration)
Daniel Holbach (dholbach) wrote :

A small thing I just noticed: please replace "bazaar" with "bzr" in debian/control.

review: Needs Fixing
Sergio Schvezov (sergiusens) wrote :

> A small thing I just noticed: please replace "bazaar" with "bzr" in
> debian/control.

revno 180 has this fix

Daniel Holbach (dholbach) wrote :

Good work!

review: Approve
Paul Larson (pwlars) wrote :

1186 + log.info('Download directory set to %s' % download_dir)
1187 + if not os.path.exists(download_dir):
1188 + log.info('Creating %s' % download_dir)
1189 + os.makedirs(download_dir)
This is still going to suffer from the makedirs race described in https://bugs.launchpad.net/phablet-tools/+bug/1209408

Otherwise, looks good.

review: Approve
Ricardo Salveti (rsalveti) wrote :

A few comments:
- Can you make --help to display all possible options by default? 'phablet-flash cdimage-touch -h' works, but 'phablet-flash -h cdimage-touch' doesn't. Also see that when you get a help from a specific group, the default ones are not part of the list (like -s and -D).

- Would it be possible to add a man page for phablet-flash as well?

- Please add a way for it to display the license when flashing a community based port. If possible I'd make this a requirement for a community build, as tag the stamp differently per device/port, so people know what they are flashing.

- Can you also remove the generated tmp files after flashing the device? Check:
INFO:phablet-flash:Pushing /tmp/tmpPNd_Gv to /cache/recovery/extendedcommand
....
rsalveti@evatp:~$ cat /tmp/tmpPNd_Gv
mount("/sdcard/");
install_zip("/sdcard/saucy-preinstalled-touch-armel+manta.zip");
install_zip("/sdcard/saucy-preinstalled-touch-armhf.zip");

Code:
721 + parser.add_argument('-d', '--device', required=True,
722 + help='Specify device to flash')

Mind adding a better help statement here? You could also link at a wiki or somewhere explaining how people can look for the supported devices. Later on we could also have a --list option to show all the possible devices to flash.

review: Needs Fixing
Ricardo Salveti (rsalveti) wrote :

Other than that, it works as expected. Flashed all the different images (for community I just downloaded it as I don't have the device), and didn't find any issue.

Would just need a major announcement as the script arguments are quite different from the previous one.

Sergio Schvezov (sergiusens) wrote :

> A few comments:
> - Can you make --help to display all possible options by default? 'phablet-
> flash cdimage-touch -h' works, but 'phablet-flash -h cdimage-touch' doesn't.
> Also see that when you get a help from a specific group, the default ones are
> not part of the list (like -s and -D).

Done, it's no imported as a parent to each command, revno 186 and 187

> - Would it be possible to add a man page for phablet-flash as well?

Can you add this as a bug so I can work on?

> - Please add a way for it to display the license when flashing a community
> based port. If possible I'd make this a requirement for a community build, as
> tag the stamp differently per device/port, so people know what they are
> flashing.

Ok, preliminary support added. revno 188

> - Can you also remove the generated tmp files after flashing the device?
> Check:
> INFO:phablet-flash:Pushing /tmp/tmpPNd_Gv to /cache/recovery/extendedcommand
> ....
> rsalveti@evatp:~$ cat /tmp/tmpPNd_Gv
> mount("/sdcard/");
> install_zip("/sdcard/saucy-preinstalled-touch-armel+manta.zip");
> install_zip("/sdcard/saucy-preinstalled-touch-armhf.zip");

revno 189 deletes the file.

> Code:
> 721 + parser.add_argument('-d', '--device', required=True,
> 722 + help='Specify device to flash')
>
> Mind adding a better help statement here? You could also link at a wiki or
> somewhere explaining how people can look for the supported devices. Later on
> we could also have a --list option to show all the possible devices to flash.

added a link to http://..../Devices (not full link to anchor as it may be an unstable link) in revno 190

Sergio Schvezov (sergiusens) wrote :

> 1186 + log.info('Download directory set to %s' % download_dir)
> 1187 + if not os.path.exists(download_dir):
> 1188 + log.info('Creating %s' % download_dir)
> 1189 + os.makedirs(download_dir)
> This is still going to suffer from the makedirs race described in
> https://bugs.launchpad.net/phablet-tools/+bug/1209408
>
> Otherwise, looks good.

I'll fix this in the next MR.

Ricardo Salveti (rsalveti) wrote :

Looks good, thanks!

review: Approve
PS Jenkins bot (ps-jenkins) wrote :

FAILED: Autolanding.
Approved revid is not set in launchpad. This is most likely a launchpad issue and re-approve should fix it. There is also a chance (although a very small one) this is a permission problem of the ps-jenkins bot.
http://jenkins.qa.ubuntu.com/job/phablet-tools-autolanding/52/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-autolanding/27
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-autolanding/27
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-autolanding/27

review: Needs Fixing (continuous-integration)
review: Approve (continuous-integration)

Preview Diff

1=== modified file 'debian/changelog'
2--- debian/changelog 2013-08-06 00:01:55 +0000
3+++ debian/changelog 2013-08-09 01:41:37 +0000
4@@ -1,3 +1,12 @@
5+phablet-tools (1.0-0ubuntu1) UNRELEASED; urgency=low
6+
7+ * phablet-flash:
8+ - Refactoring to accommodate multiple provisioning scenarios.
9+ - Command line switched into using positional arguments.
10+ - Adding support to flash community devices.
11+
12+ -- Sergio Schvezov <sergio.schvezov@canonical.com> Thu, 08 Aug 2013 08:19:54 -0300
13+
14 phablet-tools (0.15+13.10.20130806-0ubuntu1) saucy; urgency=low
15
16 [ Andy Doan ]
17
18=== modified file 'debian/control'
19--- debian/control 2013-07-14 18:46:18 +0000
20+++ debian/control 2013-08-09 01:41:37 +0000
21@@ -25,6 +25,7 @@
22 Depends:
23 android-tools-adb (>= 4.2.2),
24 android-tools-fastboot (>= 4.2.2),
25+ bzr,
26 curl,
27 python-configobj,
28 python-launchpadlib,
29
30=== modified file 'phablet-flash'
31--- phablet-flash 2013-07-30 17:17:05 +0000
32+++ phablet-flash 2013-08-09 01:41:37 +0000
33@@ -1,479 +1,64 @@
34 #! /usr/bin/python2.7
35-# This program is free software: you can redistribute it and/or modify it
36-# under the terms of the the GNU General Public License version 3, as
37-# published by the Free Software Foundation.
38-#
39-# This program is distributed in the hope that it will be useful, but
40-# WITHOUT ANY WARRANTY; without even the implied warranties of
41-# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
42-# PURPOSE. See the applicable version of the GNU Lesser General Public
43-# License for more details.
44-#.
45+# Copyright (C) 2013 Canonical Ltd.
46+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
47+
48+# This program is free software: you can redistribute it and/or modify
49+# it under the terms of the GNU General Public License as published by
50+# the Free Software Foundation; version 3 of the License.
51+#
52+# This program is distributed in the hope that it will be useful,
53+# but WITHOUT ANY WARRANTY; without even the implied warranty of
54+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
55+# GNU General Public License for more details.
56+#
57 # You should have received a copy of the GNU General Public License
58 # along with this program. If not, see <http://www.gnu.org/licenses/>.
59-#
60-# Copyright (C) 2013 Canonical, Ltd.
61
62-import argparse
63 import logging
64 import os
65-import re
66-import requests
67-import subprocess
68-import tempfile
69-import gzip
70-from os import path
71-from time import sleep
72+import sys
73+
74 from phabletutils.device import (AndroidBridge, Fastboot)
75-from phabletutils import downloads
76-from phabletutils import environment
77-from phabletutils import cdimage
78+from phabletutils import arguments
79+from phabletutils import license
80 from phabletutils import settings
81-from phabletutils import ubuntuimage
82
83-logging.basicConfig(level=logging.INFO, format='%(message)s')
84+logging.basicConfig(level=logging.INFO)
85 log = logging.getLogger()
86-log.name = 'phablet-deploy'
87-
88-
89-def parse_arguments():
90- '''Parses arguments passed in to script.'''
91- parser = argparse.ArgumentParser(
92- description='''phablet flash tool.
93- Grabs build from the network and deploys to device.
94- Does best effort to deploy in different ways.''')
95- parser.add_argument('-d',
96- '--device',
97- help='''Target device to deploy.''',
98- required=False,
99- choices=settings.supported_devices,
100- )
101- parser.add_argument('-s',
102- '--serial',
103- help='''Device serial. Use when more than
104- one device is connected.''',
105- )
106- parser.add_argument('--alternate-settings',
107- help='''Alternate default settings (not common)''',
108- )
109- parser.add_argument('--no-device-validate',
110- action='store_true',
111- default=False,
112- help='''Skip device validation, use at risk.''',
113- )
114- parser.add_argument('-b',
115- '--bootstrap',
116- help='''Bootstrap the target device, this only
117- works on Nexus devices or devices that
118- use fastboot and are unlocked. All user
119- data is destroyed''',
120- action='store_true',
121- )
122- parser.add_argument('-D',
123- '--download-only',
124- help='''Download image only, but do not flash device.
125- Use -d to override target device autodetection
126- if target device is not connected.''',
127- action='store_true',
128- )
129- parser.add_argument('--wipe',
130- help='''Cleans up all data.''',
131- action='store_true',
132- )
133- parser.add_argument('--legacy',
134- action='store_true',
135- default=False,
136- help='''Installs the legacy images'''
137- )
138- parser.add_argument('--list-revisions',
139- action='store_true',
140- required=False,
141- default=False,
142- help='List available revisions on cdimage and exits',
143- )
144- parser.add_argument('--series',
145- required=False,
146- default=None,
147- help='Forces a series, generally not needed',
148- )
149- group = parser.add_mutually_exclusive_group()
150- group.add_argument('-r',
151- '--revision',
152- required=False,
153- default=None,
154- help='''Choose a specific release to install from
155- cdimage, the format is [series]/[rev].
156- However for ubuntu-bootstrap it's a
157- relative number 0 being latest -1 being
158- the previous version and so on''',
159- )
160- group.add_argument('-l',
161- '--latest-revision',
162- action='store_true',
163- required=False,
164- help='''Pulls the latest tagged revision.''',
165- )
166- group.add_argument('-p',
167- '--base-path',
168- required=False,
169- default=None,
170- help='''Installs from base path, you must have the
171- same directory structure as if you downloaded
172- for real.
173- This option is completely offline.'''
174- )
175- group.add_argument('-u',
176- '--uri',
177- required=False,
178- help='Alternate download uri',
179- )
180- group.add_argument('--pending',
181- action='store_true',
182- required=False,
183- help='Get pending link from cdimage',
184- )
185- parser.add_argument('--ubuntu-bootstrap',
186- action='store_true',
187- required=False,
188- help='''Flash the image based upgrade Ubuntu system.
189- This action wipes the system'''
190- )
191- return parser.parse_args()
192-
193-
194-# Creates a pathname for user's answer. Touch this file.
195+log.name = 'phablet-flash'
196+
197+
198 def accepted_pathname():
199 return os.path.expanduser(settings.accept_path)
200
201
202-def accepted(pathname):
203- '''
204- Remember that the user accepted the license.
205- '''
206- open(pathname, 'w').close()
207-
208-
209-def has_accepted(pathname):
210- '''
211- Return True iff the user accepted the license once.
212- '''
213- return os.path.exists(pathname)
214-
215-
216-def query(message):
217- '''Display end user agreement to continue with deployment.'''
218- try:
219- while True:
220- print message
221- print 'Do you accept? [yes|no]'
222- answer = raw_input().lower()
223- if answer == 'yes':
224- accepted(accepted_pathname())
225- return True
226- elif answer == 'no':
227- return False
228- except KeyboardInterrupt:
229- log.error('Interruption detected, cancelling install')
230- return False
231-
232-
233-def setup_download_directory(download_dir):
234- '''
235- Tries to create the download directory from XDG_DOWNLOAD_DIR or sets
236- an alternative one.
237-
238- Returns path to directory
239- '''
240- log.info('Download directory set to %s' % download_dir)
241- if not os.path.exists(download_dir):
242- log.info('Creating %s' % download_dir)
243- os.makedirs(download_dir)
244-
245-
246-def adb_errors(f):
247- '''Decorating adb error management.'''
248- def _adb_errors(*args, **kwargs):
249- try:
250- return f(*args, **kwargs)
251- except subprocess.CalledProcessError as e:
252- log.error('Error while executing %s' %
253- e.cmd)
254- log.info('Make sure the device is connected and viewable '
255- 'by running \'adb devices\'')
256- log.info('Ensure you have a root device, one which running '
257- '\'adb root\' does not return an error')
258- exit(1)
259- return _adb_errors
260-
261-
262-@adb_errors
263-def create_recovery_file(adb, device_img, ubuntu_file):
264- template = settings.recovery_script_template
265- # Setup recovery rules
266- recovery_file = tempfile.NamedTemporaryFile(delete=False)
267- # Find out version
268- current_version = adb.getprop('ro.modversion')
269- if current_version.startswith('10.1'):
270- log.debug('Updating from multi user setup')
271- sdcard_path = '/sdcard/0'
272- else:
273- log.info('Updating from non multi user setup')
274- sdcard_path = '/sdcard'
275- recovery_script = template.format(
276- sdcard_path,
277- path.basename(device_img),
278- path.basename(ubuntu_file))
279- with recovery_file as output_file:
280- output_file.write(recovery_script)
281- log.info('Setting up recovery rules')
282- return recovery_file.name
283-
284-
285-def create_ubuntu_command_file(cache_dir, files):
286- ubuntu_command_path = os.path.join(cache_dir, 'ubuntu_command')
287- with open(ubuntu_command_path, 'w+') as output_file:
288- output_file.write(settings.ubuntu_recovery_script)
289- for f in files:
290- filename = os.path.basename(f['filename'])
291- signame = os.path.basename(f['signame'])
292- output_file.write('update %s %s\n' % (filename, signame))
293- output_file.write('unmount system\n')
294- return ubuntu_command_path
295-
296-
297-@adb_errors
298-def autodeploy(adb, artifact):
299- ''' Pushes artifact to devices sdcard and deploys'''
300- if not artifact:
301- return
302- # Can't wait-for-device here
303- sleep(15)
304- wipe_device(adb)
305- adb.push(artifact, '/sdcard/autodeploy.zip')
306- log.info('Deploying Ubuntu')
307- adb.reboot(recovery=True)
308- log.info('Installation will complete soon and reboot into Ubuntu')
309-
310-
311-def gunzip(file_path):
312- if not file_path.endswith('.gz'):
313- return file_path
314- target_path = file_path[:-3]
315- log.info('Decompressing %s' % file_path)
316- with open(target_path, 'wb') as target_file:
317- with gzip.open(file_path, 'r') as gzip_file:
318- for chunk in gzip_file.read():
319- target_file.write(chunk)
320- return target_path
321-
322-
323-def wipe_device(adb):
324- log.info('Clearing /data and /cache')
325- adb.shell('rm -Rf /cache/* /data/* /data/.developer_mode')
326- adb.shell('mkdir /cache/recovery')
327- adb.shell('mkdir /data/media')
328-
329-
330-@adb_errors
331-def deploy_recovery_image(adb, device_zip, ubuntu_zip, recovery_file,
332- wipe=False):
333- ''''
334- Deploys recovery files, recovery script and then reboots to install.
335- '''
336- if wipe:
337- wipe_device(adb)
338- adb.push(device_zip, '/sdcard/')
339- adb.push(ubuntu_zip, '/sdcard/')
340- adb.push(recovery_file, '/cache/recovery/command')
341- adb.reboot(recovery=True)
342- log.info('Once completed the device should reboot into Ubuntu')
343-
344-
345-@adb_errors
346-def detect_device(adb, device=None):
347- '''If no argument passed determine them from the connected device.'''
348- # Check CyanogenMod property
349- if not device:
350- device = adb.getprop('ro.cm.device').strip()
351- # Check Android property
352- if not device:
353- device = adb.getprop('ro.product.device').strip()
354- log.info('Device detected as %s' % device)
355- # property may not exist or we may not map it yet
356- if device not in settings.supported_devices:
357- log.error('Unsupported device, autodetect fails device')
358- log.info('When working on flipped images, detection does not '
359- 'work and would require -d')
360- exit(1)
361- return device
362-
363-
364-@adb_errors
365-def bootstrap(adb, fastboot, system_img, boot_img, recovery_img):
366- '''
367- Deploys device file using fastboot and launches recovery for ubuntu
368- deployment.
369- '''
370- adb.reboot(bootloader=True)
371- log.warning('The device needs to be unlocked for the following to work')
372-
373- fastboot.flash('system', gunzip(system_img))
374- fastboot.flash('boot', boot_img)
375- if recovery_img:
376- fastboot.flash('recovery', recovery_img)
377- fastboot.boot(recovery_img)
378- else:
379- log.info('Successfully flashed, now rebooting device')
380- fastboot.reboot()
381-
382-
383-@adb_errors
384-def validate_device(adb):
385- '''
386- Validates if the image would be installable on the target
387- '''
388- df = adb.shell('df -h').split('\n')
389- try:
390- free_data = map(str.split,
391- filter(lambda s: '/data' in s, df))[0][2]
392- except IndexError:
393- log.error('Cannot find /data mountpoint')
394- adb.reboot()
395- exit(1)
396- if free_data[-1:] == 'G' and float(free_data[:-1]) >= 3:
397- log.info('Storage requirements in /data satisfied')
398- else:
399- log.error('Not enough space in /data, found %s, rebooting', free_data)
400- adb.reboot()
401- exit(1)
402-
403-
404-@adb_errors
405-def setup_ubuntu_image_update(device, revision=-1):
406- '''
407- Sets up as described in https://wiki.ubuntu.com/ImageBasedUpgrades.
408- '''
409- # This is temporary until more commonality is found.
410- json_latest = ubuntuimage.get_json_from_index(device, revision)
411- download_dir = environment.get_download_dir_full_path(
412- os.path.join(settings.download_dir,'imageupdates',
413- str(json_latest['version'])))
414- setup_download_directory(download_dir)
415- files = ubuntuimage.download_images(download_dir, json_latest)
416- ubuntu_command_path = create_ubuntu_command_file(download_dir,
417- files['updates'])
418- return files, ubuntu_command_path
419-
420-
421-def deploy_ubuntu_image_update(adb, fastboot, recovery_img, files,
422- ubuntu_command_path):
423- adb.reboot(recovery=True)
424- wipe_device(adb)
425- for key in files:
426- for entry in files[key]:
427- adb.push(entry['filename'], '/cache/recovery/')
428- adb.push(entry['signame'], '/cache/recovery/')
429- adb.push(ubuntu_command_path, '/cache/recovery/ubuntu_command')
430- adb.reboot(bootloader=True)
431- fastboot.flash('recovery', recovery_img)
432- fastboot.boot(recovery_img)
433-
434-
435-def main(args):
436- if args.legacy and args.pending:
437- log.error('Cannot use legacy and pending together')
438- exit(1)
439- if args.ubuntu_bootstrap and args.revision:
440- ubuntu_revision = int(args.revision) - 1
441- if ubuntu_revision >= 0:
442- log.error('Revision must be 0 for current or negative number')
443- exit(1)
444- args.revision = None
445- else:
446- ubuntu_revision = -1
447- if args.list_revisions:
448- # Easy hack to get rid of the logger and inhibit requests from logging
449- log.setLevel(logging.FATAL)
450- env = environment.Environment(args.uri, None, args.series,
451- args.latest_revision, args.revision, args.pending, args.legacy,
452- args.base_path, settings)
453- env.project.list_revisions()
454- exit(0)
455- if not has_accepted(accepted_pathname()) and \
456- not query(settings.legal_notice):
457- exit(1)
458- adb = AndroidBridge(args.serial)
459- fastboot = Fastboot(args.serial)
460- # Initializes the adb server if it is not running
461- adb.start()
462- device = detect_device(adb, args.device)
463- try:
464- env = environment.Environment(args.uri, device, args.series,
465- args.latest_revision, args.revision, args.pending, args.legacy,
466- args.base_path, settings)
467- except EnvironmentError as e:
468- log.error(e.message)
469- exit(1)
470- log.info('Download uri set to %s' % env.download_uri)
471- if env.download_uri:
472- setup_download_directory(env.download_dir)
473- download_list = []
474- if args.ubuntu_bootstrap:
475- download_list.append(env.recovery_img_path)
476- elif args.bootstrap:
477- [download_list.append(f) for f in env.bootstrap_files]
478- else:
479- [download_list.append(f) for f in env.recovery_files]
480- try:
481- if env.project:
482- hashes = env.project.hashes
483- else:
484- hashes = None
485- log.info('Retrieving files')
486- downloader = downloads.Downloader(env.download_uri,
487- download_list, hashes)
488- downloader.download()
489- except KeyboardInterrupt:
490- log.info('To continue downloading in the future, rerun the same '
491- 'command')
492- exit(1)
493- except EnvironmentError as e:
494- log.error(e.message)
495- exit(1)
496- except subprocess.CalledProcessError:
497- log.error('Error while downloading, ensure connection')
498- exit(1)
499- if env.download_uri and env.project:
500- env.store_hashes()
501- if not args.download_only:
502- if args.ubuntu_bootstrap:
503- ubuntu_files, ubuntu_command_path = \
504- setup_ubuntu_image_update(device, ubuntu_revision)
505- deploy_ubuntu_image_update(adb, fastboot, env.recovery_img_path,
506- ubuntu_files, ubuntu_command_path)
507- elif args.bootstrap:
508- bootstrap(adb, fastboot, env.system_img_path,
509- env.boot_img_path, env.recovery_img_path)
510- autodeploy(adb, env.ubuntu_zip_path)
511- else:
512- adb.reboot(recovery=True)
513- validate_device(adb)
514- recovery_file = create_recovery_file(adb,
515- env.device_zip_path, env.ubuntu_zip_path)
516- deploy_recovery_image(adb, env.device_zip_path,
517- env.ubuntu_zip_path, recovery_file, args.wipe)
518-
519-
520-def import_alt_settings(alternate_settings):
521- import imp
522- global settings
523- dirname, basename = os.path.split(alternate_settings)
524- f, filename, desc = imp.find_module(basename.rstrip('\.py'), [dirname])
525- settings = imp.load_module(basename.rstrip('\.py'), f, filename, desc)
526+def main(argv):
527+ parser = arguments.get_parser()
528+ args = parser.parse_args(argv[1:])
529+ if args.debug:
530+ log.setLevel(logging.DEBUG)
531+ if not license.has_accepted(accepted_pathname()) and \
532+ not license.query(settings.legal_notice, accepted_pathname()):
533+ exit(1)
534+ try:
535+ project = args.func(args)
536+ if project:
537+ fastboot = Fastboot(args.serial)
538+ adb = AndroidBridge(args.serial)
539+ adb.start()
540+ project.download()
541+ if not args.download_only:
542+ project.install(adb, fastboot)
543+ except KeyboardInterrupt:
544+ log.info('Provisioning manually interrupted. Resume by rerunning '
545+ 'the command')
546+ exit(1)
547+ except Exception as e:
548+ log.error(e)
549+ if args.debug:
550+ log.exception(e)
551+ exit(1)
552
553
554 if __name__ == "__main__":
555- args = parse_arguments()
556- if args.alternate_settings:
557- import_alt_settings(args.alternate_settings)
558- main(args)
559+ main(sys.argv)
560
561=== modified file 'phabletutils/__init__.py'
562--- phabletutils/__init__.py 2013-01-16 14:55:09 +0000
563+++ phabletutils/__init__.py 2013-08-09 01:41:37 +0000
564@@ -1,5 +1,6 @@
565 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
566 # Copyright 2013 Canonical
567+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
568 #
569 # This program is free software: you can redistribute it and/or modify it
570 # under the terms of the GNU General Public License version 3, as published
571
572=== added file 'phabletutils/arguments.py'
573--- phabletutils/arguments.py 1970-01-01 00:00:00 +0000
574+++ phabletutils/arguments.py 2013-08-09 01:41:37 +0000
575@@ -0,0 +1,256 @@
576+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
577+# Copyright (C) 2013 Canonical Ltd.
578+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
579+
580+# This program is free software: you can redistribute it and/or modify
581+# it under the terms of the GNU General Public License as published by
582+# the Free Software Foundation; version 3 of the License.
583+#
584+# This program is distributed in the hope that it will be useful,
585+# but WITHOUT ANY WARRANTY; without even the implied warranty of
586+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
587+# GNU General Public License for more details.
588+#
589+# You should have received a copy of the GNU General Public License
590+# along with this program. If not, see <http://www.gnu.org/licenses/>.
591+
592+import argparse
593+import logging
594+import tempfile
595+import urllib
596+import urlparse
597+
598+from phabletutils import cdimage
599+from phabletutils import environment
600+from phabletutils import resources
601+from phabletutils import settings
602+
603+log = logging.getLogger()
604+
605+
606+class PathAction(argparse.Action):
607+ def __call__(self, parser, namespace, values, option_string=None):
608+ log.debug('PathAction: %r %r %r' %
609+ (namespace, values, option_string))
610+ uri = urlparse.urlparse(values)
611+ if not uri.scheme or uri.scheme == 'file':
612+ if '%' in uri.path:
613+ zip_file_path = urllib.unquote(
614+ urlparse.urlparse(uri.path))
615+ else:
616+ zip_file_path = uri.path
617+ zip_file_uri = None
618+ check = False
619+ elif uri.scheme == 'http' or uri.scheme == 'https':
620+ zip_file_path = tempfile.mktemp()
621+ zip_file_uri = values
622+ check = True
623+ log.debug('Download from %s, path on disk %s' %
624+ (zip_file_uri, zip_file_path))
625+ artifact = resources.File(file_path=zip_file_path,
626+ file_uri=zip_file_uri,
627+ check=check)
628+ setattr(namespace, self.dest, artifact)
629+
630+
631+class RevisionListAction(argparse.Action):
632+ def __call__(self, parser, namespace, values, option_string=None):
633+ log.debug('RevisionListAction: %r %r %r' %
634+ (namespace, values, option_string))
635+ project = 'ubuntu-touch-preview'
636+ uri = '%s/%s' % (settings.cdimage_uri_base, project)
637+ setattr(namespace, 'func', environment.list_revisions)
638+ setattr(namespace, 'uri', uri)
639+
640+
641+class RevisionAction(argparse.Action):
642+ def __call__(self, parser, namespace, values, option_string=None):
643+ log.debug('RevisionAction: %r %r %r' %
644+ (namespace, values, option_string))
645+ project = 'ubuntu-touch-preview'
646+ base_uri = '%s/%s' % (settings.cdimage_uri_base, project)
647+ if values:
648+ revision_split = values.split('/')
649+ if len(revision_split) != 2:
650+ raise EnvironmentError(
651+ 'Improper use of revision, needs to be formatted like'
652+ '[series]/[revision]. Use --list-revisions to find'
653+ 'the available revisions on cdimage')
654+ series = revision_split[0]
655+ build = revision_split[1]
656+ else:
657+ series, build = cdimage.get_latest_revision(base_uri)
658+ uri = '%s/%s/%s' % (base_uri, series, build)
659+ setattr(namespace, self.dest, True)
660+ setattr(namespace, 'series', series)
661+ setattr(namespace, 'build', build)
662+ setattr(namespace, 'uri', uri)
663+
664+
665+def cdimage_touch(parent_parser, parents):
666+ parser = parent_parser.add_parser(
667+ 'cdimage-touch', parents=parents,
668+ help='Provisions the device with a CDimage build of Ubuntu Touch.')
669+ parser.set_defaults(func=environment.setup_cdimage_touch,
670+ project='ubuntu-touch',
671+ series=settings.default_series,
672+ build=None,
673+ uri=None)
674+ parser.add_argument('-b',
675+ '--bootstrap',
676+ help='''Bootstrap the target device, this only
677+ works on Nexus devices or devices that
678+ use fastboot and are unlocked. All user
679+ data is destroyed''',
680+ action='store_true')
681+ group = parser.add_mutually_exclusive_group()
682+ group.add_argument('--pending',
683+ action='store_true',
684+ required=False,
685+ help='Get pending link from cdimage')
686+ group.add_argument('-p',
687+ '--base-path',
688+ required=False,
689+ default=None,
690+ help='''Installs from base path, you must have
691+ the same directory structure as if you
692+ downloaded for real.
693+ This option is completely offline.''')
694+ return parser
695+
696+
697+def ubuntu_system(parent_parser, parents):
698+ parser = parent_parser.add_parser(
699+ 'ubuntu-system', parents=parents,
700+ help='Provisions the device with an Ubuntu Image Based Upgrade image.')
701+ parser.set_defaults(func=environment.setup_ubuntu_system,
702+ revision=0,
703+ series=settings.default_series,
704+ project='imageupdates')
705+ parser.add_argument('--revision',
706+ type=int,
707+ help='''Download a relative revision from current
708+ (-1, -2, ...) or a specific version.''')
709+ return parser
710+
711+
712+def legacy(parent_parser, parents):
713+ parser = parent_parser.add_parser(
714+ 'cdimage-legacy', parents=parents,
715+ help='Provisions the device with legacy unflipped images.')
716+ parser.set_defaults(func=environment.setup_cdimage_legacy,
717+ project='ubuntu-touch-preview',
718+ series=settings.default_series,
719+ build=None,
720+ uri=None)
721+ parser.add_argument('-b',
722+ '--bootstrap',
723+ help='''Bootstrap the target device, this only
724+ works on Nexus devices or devices that
725+ use fastboot and are unlocked. All user
726+ data is destroyed''',
727+ action='store_true')
728+ group = parser.add_mutually_exclusive_group()
729+ group.add_argument('--list-revisions',
730+ action=RevisionListAction,
731+ nargs=0,
732+ help='List available revisions on cdimage and exits')
733+ group.add_argument('-r',
734+ '--revision',
735+ action=RevisionAction,
736+ help='''Choose a specific release to install
737+ from cdimage, the format is
738+ [series]/[rev].''')
739+ group.add_argument('-l',
740+ '--latest-revision',
741+ action=RevisionAction,
742+ nargs=0,
743+ help='''Pulls the latest tagged revision.''')
744+ group.add_argument('-p',
745+ '--base-path',
746+ required=False,
747+ default=None,
748+ help='''Installs from base path, you must have
749+ the same directory structure as if you
750+ downloaded for real.
751+ This option is completely offline.''')
752+ return parser
753+
754+
755+def community(parent_parser, parents):
756+ parser = parent_parser.add_parser(
757+ 'community', parents=parents,
758+ help='Provisions the device with a community supported build.')
759+ parser.add_argument('-d', '--device', required=True,
760+ help='''Specify device to flash.
761+ Find out more about flashable devices at
762+ https://wiki.ubuntu.com/Touch/Devices''')
763+ parser.set_defaults(func=environment.setup_community,
764+ series=settings.default_series)
765+ return parser
766+
767+
768+def common_non_system():
769+ parser = argparse.ArgumentParser(add_help=False)
770+ parser.add_argument('--device-path',
771+ action=PathAction,
772+ help='''uri to device zip to flash,
773+ e.g.; file:///..., http://.''',)
774+ parser.add_argument('--ubuntu-path',
775+ action=PathAction,
776+ help='''uri to ubuntu zip to flash,
777+ e.g.; file:///..., http://.''',)
778+ parser.add_argument('--wipe',
779+ action='store_true',
780+ help='''Wipes device data.''')
781+ return parser
782+
783+
784+def common_supported():
785+ parser = argparse.ArgumentParser(add_help=False)
786+ parser.add_argument('-d',
787+ '--device',
788+ help='''Target device to deploy.''',
789+ required=False,
790+ choices=settings.supported_devices)
791+ return parser
792+
793+
794+def common():
795+ parser = argparse.ArgumentParser(add_help=False)
796+ parser.add_argument('--debug',
797+ action='store_true',
798+ help='''Enable debug messages.''')
799+ parser.add_argument('-s',
800+ '--serial',
801+ help='''Device serial. Use when more than
802+ one device is connected.''')
803+ parser.add_argument('-D',
804+ '--download-only',
805+ action='store_true',
806+ help='Download image only, but do not flash device.')
807+ return parser
808+
809+
810+def get_parser():
811+ """
812+ Returns a Namespace of the parsed arguments from the command line.
813+ """
814+ parser = argparse.ArgumentParser(
815+
816+ description='''phablet-flash is used to provision devices.
817+ It does a best effort to deploy in different ways.''',
818+ epilog='''Use -h or --help after each command to learn about
819+ their provisioning options.''')
820+ # Parsers
821+ common_parser = common()
822+ common_supported_parser = common_supported()
823+ common_non_system_parser = common_non_system()
824+ sub = parser.add_subparsers(title='Commands', metavar='')
825+ cdimage_touch(sub, [common_parser, common_supported_parser,
826+ common_non_system_parser])
827+ legacy(sub, [common_parser, common_supported_parser,
828+ common_non_system_parser])
829+ ubuntu_system(sub, [common_supported_parser, ])
830+ community(sub, [common_parser, common_non_system_parser, ])
831+ return parser
832
833=== modified file 'phabletutils/cdimage.py'
834--- phabletutils/cdimage.py 2013-07-11 22:28:31 +0000
835+++ phabletutils/cdimage.py 2013-08-09 01:41:37 +0000
836@@ -1,24 +1,29 @@
837 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
838-# Copyright 2013 Canonical
839-#
840-# This program is free software: you can redistribute it and/or modify it
841-# under the terms of the GNU General Public License version 3, as published
842-# by the Free Software Foundation.
843-#
844-# This program is distributed in the hope that it will be useful, but
845-# WITHOUT ANY WARRANTY; without even the implied warranties of
846-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
847-# PURPOSE. See the GNU General Public License for more details.
848+# Copyright (C) 2013 Canonical Ltd.
849+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
850
851-# You should have received a copy of the GNU General Public License along
852-# with this program. If not, see <http://www.gnu.org/licenses/>.
853+# This program is free software: you can redistribute it and/or modify
854+# it under the terms of the GNU General Public License as published by
855+# the Free Software Foundation; version 3 of the License.
856+#
857+# This program is distributed in the hope that it will be useful,
858+# but WITHOUT ANY WARRANTY; without even the implied warranty of
859+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
860+# GNU General Public License for more details.
861+#
862+# You should have received a copy of the GNU General Public License
863+# along with this program. If not, see <http://www.gnu.org/licenses/>.
864
865 import logging
866 import re
867 import requests
868+import os.path
869 import subprocess
870 import urlparse
871
872+from phabletutils import hashes
873+from phabletutils import resources
874+from phabletutils import settings
875
876 log = logging.getLogger()
877
878@@ -95,8 +100,6 @@
879 print 'No releases for %s available' % rev['release']
880 if not revisions:
881 print 'No revisions have been tagged for this project yet'
882- print 'To list revisions from the preview images add the ' \
883- '--legacy switch'
884
885
886 def _get_link_target(uri):
887@@ -107,40 +110,27 @@
888 return link
889
890
891-def get_latest_current(cdimage_uri):
892+def get_build(cdimage_uri, pending=False):
893 '''Returns the latest build in current.'''
894- uri = '%s/%s' % (cdimage_uri, 'current')
895- build = _get_link_target(uri)
896- return build
897-
898-
899-def get_latest_pending(cdimage_uri):
900- '''Returns the latest build in pending.'''
901- uri = '%s/%s' % (cdimage_uri, 'pending')
902- build = _get_link_target(uri)
903- return build
904-
905-
906-def get_sha256_dict(hash_file_content):
907- '''Returns a dictionary with the sha256 sums for all files.'''
908- if not hash_file_content:
909- log.debug('hash file is empty')
910- return None
911- hash_list = filter((lambda x: len(x) is not 0),
912- hash_file_content.split('\n'))
913- hash_list = [h.split() for h in hash_list]
914- hash_dict = {}
915- for hash_entry in hash_list:
916- if hash_entry[1][0] == '*':
917- hash_entry[1] = hash_entry[1][1:]
918- hash_dict[hash_entry[1]] = hash_entry[0]
919- return hash_dict
920-
921-
922-def get_sha256_content(cdimage_hash_uri):
923- '''Fetches the SHA256 sum file from cdimage.'''
924- hash_request = requests.get(cdimage_hash_uri)
925- if hash_request.status_code != 200:
926- return None
927- else:
928- return hash_request.content
929+ if pending:
930+ uri = '%s/%s' % (cdimage_uri, 'pending')
931+ else:
932+ uri = '%s/%s' % (cdimage_uri, 'current')
933+ build = _get_link_target(uri)
934+ return build
935+
936+
937+def get_file(file_key, series, download_dir, project='ubuntu-touch',
938+ device=None):
939+ uri = '%s/%s/daily-preinstalled/current' % (
940+ settings.cdimage_uri_base, project)
941+ hash_dict = hashes.load_hash(uri, 'SHA256SUMS')
942+ if device:
943+ file_name = settings.files_arch_any[project][file_key] %\
944+ (series, device)
945+ else:
946+ file_name = settings.files_arch_all[project][file_key] % series
947+ return resources.File(
948+ file_path=os.path.join(download_dir, file_name),
949+ file_uri='%s/%s' % (uri, file_name),
950+ file_hash=hash_dict[file_name])
951
952=== added file 'phabletutils/community.py'
953--- phabletutils/community.py 1970-01-01 00:00:00 +0000
954+++ phabletutils/community.py 2013-08-09 01:41:37 +0000
955@@ -0,0 +1,129 @@
956+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
957+# Copyright (C) 2013 Canonical Ltd.
958+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
959+
960+# This program is free software: you can redistribute it and/or modify
961+# it under the terms of the GNU General Public License as published by
962+# the Free Software Foundation; version 3 of the License.
963+#
964+# This program is distributed in the hope that it will be useful,
965+# but WITHOUT ANY WARRANTY; without even the implied warranty of
966+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
967+# GNU General Public License for more details.
968+#
969+# You should have received a copy of the GNU General Public License
970+# along with this program. If not, see <http://www.gnu.org/licenses/>.
971+
972+from __future__ import print_function
973+
974+import json
975+import hashlib
976+import logging
977+import os
978+import os.path
979+import subprocess
980+
981+from phabletutils import cdimage
982+from phabletutils import downloads
983+from phabletutils import resources
984+from phabletutils import license
985+from phabletutils import settings
986+
987+log = logging.getLogger()
988+branch_template = 'lp:~{0}-image-dev/phablet-image-info/{0}'
989+base_dir = os.path.join(settings.download_dir, 'community')
990+
991+license_template = '''This community project has the following license:
992+
993+{0}
994+'''
995+
996+
997+def branch_project(device):
998+ branch = branch_template.format(device.replace('_', '-'))
999+ log.info('Obtaining project branch from %s' % branch)
1000+ download_dir = downloads.get_full_path(os.path.join(base_dir, device))
1001+ target = os.path.join(download_dir, 'config')
1002+ if os.path.exists(target):
1003+ subprocess.check_call(['bzr', 'update'], cwd=target)
1004+ else:
1005+ subprocess.check_call(
1006+ ['bzr', 'checkout', '--lightweight', branch, target])
1007+ log.info('Target config retrieved to %s' % target)
1008+ # Best time to do this is right after retrieving the config
1009+ ensure_license_accept(download_dir, os.path.join(target, 'license'))
1010+ return target
1011+
1012+
1013+def ensure_license_accept(download_dir, license_file):
1014+ if not os.path.exists(license_file):
1015+ raise EnvironmentError('Project does not offer a license file')
1016+ accept_path = os.path.join(download_dir, '.license_accept')
1017+ with open(license_file, 'r') as f:
1018+ device_license = f.read()
1019+ device_license = 'LICENSE TEXT NOT PROVIDED BY PORT MAINTAINER' \
1020+ if not device_license else device_license
1021+ message = license_template.format(device_license)
1022+ if not license.has_accepted(accept_path) and \
1023+ not license.query(message, accept_path):
1024+ raise RuntimeError('License not accepted.')
1025+
1026+
1027+def load_manifest(directory):
1028+ manifest_file = os.path.join(directory, 'manifest.json')
1029+ if not os.path.exists(manifest_file):
1030+ raise RuntimeError('Cannot locate %s' % manifest_file)
1031+ with open(manifest_file) as f:
1032+ manifest_dict = json.load(f)
1033+ if 'device' not in manifest_dict:
1034+ raise EnvironmentError('device entry required in manifest')
1035+ if 'revision' not in manifest_dict:
1036+ manifest_dict['revision'] = None
1037+ if 'storage' not in manifest_dict:
1038+ manifest_dict['storage'] = '/sdcard/'
1039+ if 'ubuntu' not in manifest_dict:
1040+ manifest_dict['ubuntu'] = None
1041+ log.debug(json.dumps(manifest_dict))
1042+ return manifest_dict
1043+
1044+
1045+def get_download_dir(device, revision=None):
1046+ download_dir = os.path.join(settings.download_dir, 'community', device)
1047+ if revision:
1048+ download_dir = os.path.join(download_dir, revision)
1049+ return downloads.get_full_path(download_dir)
1050+
1051+
1052+def get_files(manifest_dict, download_dir, series):
1053+ files = {}
1054+ for key in ('device', 'ubuntu'):
1055+ if isinstance(manifest_dict[key], dict):
1056+ item = manifest_dict[key]
1057+ hash_type = item['hash_func'] if 'hash_func' in item else None
1058+ log.debug('%s has config uri: %s' % (key, item['uri']))
1059+ files[key] = resources.File(
1060+ file_path=os.path.join(download_dir, '%s.zip' % key),
1061+ file_uri=item['uri'],
1062+ file_hash=item['hash'] if 'hash' in item else None,
1063+ file_hash_func=get_hash_func(hash_type) if hash_type else None)
1064+ elif isinstance(manifest_dict[key], str) or \
1065+ isinstance(manifest_dict[key], unicode):
1066+ log.debug('%s has config uri: %s' % (key, manifest_dict[key]))
1067+ files[key] = resources.File(
1068+ file_path=os.path.join(download_dir, '%s.zip' % key),
1069+ file_uri=manifest_dict[key],
1070+ file_hash=None)
1071+ log.debug('%s is type %s' % (key, type(manifest_dict[key])))
1072+ if not manifest_dict['ubuntu']:
1073+ log.warning('Using Ubuntu Touch current build for ubuntu image')
1074+ files['ubuntu'] = cdimage.get_file(file_key='ubuntu_zip',
1075+ series=series,
1076+ download_dir=download_dir)
1077+ return files
1078+
1079+
1080+def get_hash_func(hash_type):
1081+ return {
1082+ 'md5': hashlib.md5,
1083+ 'sha256': hashlib.sha256,
1084+ }.get(hash_type, None)
1085
1086=== modified file 'phabletutils/device.py'
1087--- phabletutils/device.py 2013-07-02 18:37:54 +0000
1088+++ phabletutils/device.py 2013-08-09 01:41:37 +0000
1089@@ -1,17 +1,18 @@
1090 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
1091-# Copyright 2013 Canonical
1092-#
1093-# This program is free software: you can redistribute it and/or modify it
1094-# under the terms of the GNU General Public License version 3, as published
1095-# by the Free Software Foundation.
1096-#
1097-# This program is distributed in the hope that it will be useful, but
1098-# WITHOUT ANY WARRANTY; without even the implied warranties of
1099-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
1100-# PURPOSE. See the GNU General Public License for more details.
1101+# Copyright (C) 2013 Canonical Ltd.
1102+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
1103
1104-# You should have received a copy of the GNU General Public License along
1105-# with this program. If not, see <http://www.gnu.org/licenses/>.
1106+# This program is free software: you can redistribute it and/or modify
1107+# it under the terms of the GNU General Public License as published by
1108+# the Free Software Foundation; version 3 of the License.
1109+#
1110+# This program is distributed in the hope that it will be useful,
1111+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1112+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1113+# GNU General Public License for more details.
1114+#
1115+# You should have received a copy of the GNU General Public License
1116+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1117
1118 import subprocess
1119 import logging
1120
1121=== modified file 'phabletutils/downloads.py'
1122--- phabletutils/downloads.py 2013-08-02 22:01:03 +0000
1123+++ phabletutils/downloads.py 2013-08-09 01:41:37 +0000
1124@@ -1,108 +1,130 @@
1125-# This program is free software: you can redistribute it and/or modify it
1126-# under the terms of the the GNU General Public License version 3, as
1127-# published by the Free Software Foundation.
1128-#
1129-# This program is distributed in the hope that it will be useful, but
1130-# WITHOUT ANY WARRANTY; without even the implied warranties of
1131-# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
1132-# PURPOSE. See the applicable version of the GNU Lesser General Public
1133-# License for more details.
1134-#.
1135+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
1136+# Copyright (C) 2013 Canonical Ltd.
1137+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
1138+
1139+# This program is free software: you can redistribute it and/or modify
1140+# it under the terms of the GNU General Public License as published by
1141+# the Free Software Foundation; version 3 of the License.
1142+#
1143+# This program is distributed in the hope that it will be useful,
1144+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1145+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1146+# GNU General Public License for more details.
1147+#
1148 # You should have received a copy of the GNU General Public License
1149 # along with this program. If not, see <http://www.gnu.org/licenses/>.
1150-#
1151-# Copyright (C) 2013 Canonical, Ltd.
1152
1153+import configobj
1154 import contextlib
1155 import fcntl
1156 import hashlib
1157 import logging
1158 import os
1159+import requests
1160 import subprocess
1161
1162+from xdg.BaseDirectory import xdg_config_home
1163+
1164
1165 log = logging.getLogger()
1166
1167
1168-def download(uri, target):
1169- '''Downloads an artifact into target.'''
1170- log.info('Downloading %s' % uri)
1171- if uri.startswith('http://cdimage.ubuntu.com') or \
1172- uri.startswith('https://system-image.ubuntu.com'):
1173- subprocess.check_call(['wget', '-c', uri, '-O', target])
1174- else:
1175- subprocess.check_call(['curl', '-L', '-C', '-', uri, '-o', target])
1176-
1177-
1178 @contextlib.contextmanager
1179 def flocked(lockfile):
1180 lockfile += '.lock'
1181 with open(lockfile, 'w') as f:
1182- log.debug('aquiring lock for %s', lockfile)
1183+ log.debug('Aquiring lock for %s', lockfile)
1184 try:
1185 fcntl.lockf(f, fcntl.LOCK_EX)
1186 yield
1187 finally:
1188+ log.debug('Releasing lock for %s', lockfile)
1189 fcntl.lockf(f, fcntl.LOCK_UN)
1190
1191
1192-class Downloader(object):
1193- '''A helper to fetch multiple artifacts.'''
1194-
1195- def __init__(self, uri, download_list, hashes):
1196- '''Initializes a download helper for android based images.'''
1197- self._uri = uri
1198- self._download_list = download_list
1199- self._hashes = hashes
1200- if hashes:
1201- self.validate = self._validate
1202- else:
1203- self.validate = self._validate_legacy
1204-
1205- def download(self):
1206- '''Downloads and validates the download list.'''
1207- for file_path in self._download_list:
1208- with flocked(file_path):
1209- if self.validate(file_path):
1210- continue
1211- uri = '%s/%s' % (self._uri, os.path.basename(file_path))
1212- if not uri:
1213- fmt = ('In offline mode and checksum does not match '
1214- 'for at least %s')
1215- raise EnvironmentError(fmt % file_path)
1216- download(uri, file_path)
1217- if not self._hashes:
1218- file_hash = '%s.md5sum'
1219- download(file_hash % uri, file_hash % file_path)
1220- if not self.validate(file_path):
1221- fmt = 'File download failed for %s to %s'
1222- raise EnvironmentError(fmt % (uri, file_path))
1223- log.info('Validating download of %s' % file_path)
1224-
1225- def _validate(self, file_path):
1226- '''Validates downloaded files against a checksum.'''
1227- file_name = os.path.basename(file_path)
1228- hashes = self._hashes()
1229- if file_name not in hashes:
1230- return False
1231- return self.checksum_file(file_path) == hashes[file_name]
1232-
1233- def _validate_legacy(self, file_path):
1234- '''Validates with the legacy method of individual md5sum files.'''
1235- file_hash = '%s.md5sum' % file_path
1236- if not os.path.exists(file_hash):
1237- return False
1238- with open(file_hash, 'r') as f:
1239- read_hash = f.read().split()[0]
1240- return self.checksum_file(file_path, hashlib.md5) == read_hash
1241-
1242- def checksum_file(self, file_path, sum_method=hashlib.sha256):
1243- '''Returns the checksum for a file with a specified algorightm.'''
1244- file_sum = sum_method()
1245- if not os.path.exists(file_path):
1246- return None
1247- with open(file_path, 'rb') as f:
1248- for file_chunk in iter(
1249- lambda: f.read(file_sum.block_size * 128), b''):
1250- file_sum.update(file_chunk)
1251- return file_sum.hexdigest()
1252+def setup_download_directory(download_dir):
1253+ '''
1254+ Tries to create the download directory from XDG_DOWNLOAD_DIR or sets
1255+ an alternative one.
1256+
1257+ Returns path to directory
1258+ '''
1259+ log.info('Download directory set to %s' % download_dir)
1260+ if not os.path.exists(download_dir):
1261+ log.info('Creating %s' % download_dir)
1262+ os.makedirs(download_dir)
1263+
1264+
1265+def get_full_path(subdir):
1266+ try:
1267+ userdirs_file = os.path.join(xdg_config_home, 'user-dirs.dirs')
1268+ userdirs_config = configobj.ConfigObj(userdirs_file, encoding='utf-8')
1269+ userdirs_download = os.path.expandvars(
1270+ userdirs_config['XDG_DOWNLOAD_DIR'])
1271+ download_dir = userdirs_download
1272+ except KeyError:
1273+ download_dir = os.path.expandvars('$HOME')
1274+ log.warning('XDG_DOWNLOAD_DIR could not be read')
1275+ directory = os.path.join(download_dir, subdir)
1276+ setup_download_directory(directory)
1277+ return directory
1278+
1279+
1280+def checksum_verify(file_path, file_hash, sum_method=hashlib.sha256):
1281+ '''Returns the checksum for a file with a specified algorightm.'''
1282+ file_sum = sum_method()
1283+ log.debug('Verifying file: %s against: %s' % (file_path, file_hash))
1284+ if not os.path.exists(file_path):
1285+ log.debug('File %s not found' % file_path)
1286+ return False
1287+ with open(file_path, 'rb') as f:
1288+ for file_chunk in iter(
1289+ lambda: f.read(file_sum.block_size * 128), b''):
1290+ file_sum.update(file_chunk)
1291+ if file_hash == file_sum.hexdigest():
1292+ return True
1293+ else:
1294+ log.debug('Calculated sum mismatch calculated %s != %s' %
1295+ (file_sum.hexdigest(), file_hash))
1296+ return False
1297+
1298+
1299+def _download(uri, path):
1300+ if uri.startswith('http://cdimage.ubuntu.com') or \
1301+ uri.startswith('https://system-image.ubuntu.com'):
1302+ subprocess.check_call(['wget',
1303+ '-c',
1304+ uri,
1305+ '-O',
1306+ path])
1307+ else:
1308+ subprocess.check_call(['curl',
1309+ '-L',
1310+ '-C',
1311+ '-',
1312+ uri,
1313+ '-o',
1314+ path])
1315+
1316+
1317+def download_sig(artifact):
1318+ '''Downloads an artifact into target.'''
1319+ log.info('Downloading %s to %s' % (artifact.uri, artifact.path))
1320+ with flocked(artifact._sig_path):
1321+ _download(artifact.sig_uri, artifact.sig_path)
1322+
1323+
1324+def download(artifact):
1325+ '''Downloads an artifact into target.'''
1326+ log.info('Downloading %s to %s' % (artifact.uri, artifact.path))
1327+ with flocked(artifact._path):
1328+ _download(artifact.uri, artifact.path)
1329+
1330+
1331+def get_content(uri):
1332+ '''Fetches the SHA256 sum file from cdimage.'''
1333+ content_request = requests.get(uri)
1334+ if content_request.status_code != 200:
1335+ return None
1336+ else:
1337+ return content_request.content
1338
1339=== modified file 'phabletutils/environment.py'
1340--- phabletutils/environment.py 2013-08-02 21:00:48 +0000
1341+++ phabletutils/environment.py 2013-08-09 01:41:37 +0000
1342@@ -1,42 +1,39 @@
1343 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
1344-# Copyright 2013 Canonical
1345-#
1346-# This program is free software: you can redistribute it and/or modify it
1347-# under the terms of the GNU General Public License version 3, as published
1348-# by the Free Software Foundation.
1349-#
1350-# This program is distributed in the hope that it will be useful, but
1351-# WITHOUT ANY WARRANTY; without even the implied warranties of
1352-# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
1353-# PURPOSE. See the GNU General Public License for more details.
1354-
1355-# You should have received a copy of the GNU General Public License along
1356-# with this program. If not, see <http://www.gnu.org/licenses/>.
1357-
1358-import configobj
1359+# Copyright (C) 2013 Canonical Ltd.
1360+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
1361+
1362+# This program is free software: you can redistribute it and/or modify
1363+# it under the terms of the GNU General Public License as published by
1364+# the Free Software Foundation; version 3 of the License.
1365+#
1366+# This program is distributed in the hope that it will be useful,
1367+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1368+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1369+# GNU General Public License for more details.
1370+#
1371+# You should have received a copy of the GNU General Public License
1372+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1373+
1374+from __future__ import print_function
1375+
1376+import hashlib
1377 import logging
1378+import os.path
1379 import requests
1380
1381-from os import path
1382+from phabletutils.device import AndroidBridge
1383 from phabletutils import cdimage
1384-from xdg.BaseDirectory import xdg_config_home
1385+from phabletutils import community
1386+from phabletutils import downloads
1387+from phabletutils import hashes
1388+from phabletutils import resources
1389+from phabletutils import projects
1390+from phabletutils import settings
1391+from phabletutils import ubuntuimage
1392
1393 log = logging.getLogger()
1394
1395
1396-def get_download_dir_full_path(subdir):
1397- try:
1398- userdirs_file = path.join(xdg_config_home, 'user-dirs.dirs')
1399- userdirs_config = configobj.ConfigObj(userdirs_file, encoding='utf-8')
1400- userdirs_download = path.expandvars(
1401- userdirs_config['XDG_DOWNLOAD_DIR'])
1402- download_dir = userdirs_download
1403- except KeyError:
1404- download_dir = path.expandvars('$HOME')
1405- log.warning('XDG_DOWNLOAD_DIR could not be read')
1406- return path.join(download_dir, subdir)
1407-
1408-
1409 def get_ubuntu_stamp(uri):
1410 '''Downloads the jenkins build id from stamp file'''
1411 try:
1412@@ -54,224 +51,187 @@
1413 log.error('Could not download build data from jenkins... bailing')
1414 exit(1)
1415 except IndexError:
1416- log.error('Jenkins data format has changed, incompatible')
1417- exit(1)
1418+ raise EnvironmentError('Jenkins data format has changed, incompatible')
1419 return jenkins_build_id
1420
1421
1422-class Project(object):
1423-
1424- @property
1425- def uri(self):
1426- return self._uri
1427-
1428- @property
1429- def download_uri(self):
1430- return self._download_uri
1431-
1432- @property
1433- def download_dir(self):
1434- return self._download_dir
1435-
1436- @property
1437- def name(self):
1438- return self._project
1439-
1440- @property
1441- def series(self):
1442- return self._series
1443-
1444- def hashes(self):
1445- return cdimage.get_sha256_dict(self.hash_content)
1446-
1447- @property
1448- def hash_content(self):
1449- self._load_hashes()
1450- return self._hash_content
1451-
1452- @property
1453- def hash_file_name(self):
1454- return self._hash_file_name
1455-
1456- def list_revisions(self):
1457- revisions = cdimage.get_available_revisions(self._uri)
1458- cdimage.display_revisions(revisions)
1459-
1460- def __init__(self, project, series, cdimage_uri_base, base_path,
1461- hash_file_name, pending, revision=None, latest_revision=None):
1462- if not project:
1463- return None
1464- log.debug('project: %s, series: %s, cdimage_uri_base: %s, '
1465- 'base_path: %s, hash_file_name: %s, revision: %s, '
1466- 'latest_revision: %s, pending: %s' % (project, series,
1467- cdimage_uri_base, base_path, hash_file_name, revision,
1468- latest_revision, pending))
1469- self._project = project
1470- self._uri = '%s/%s' % (cdimage_uri_base, project)
1471- self._series = series
1472- self._hash_file_name = hash_file_name
1473-
1474- if base_path:
1475- download_uri = None
1476- download_dir = base_path
1477- elif latest_revision:
1478- self._series, revision = \
1479- cdimage.get_latest_revision(self._uri)
1480- download_uri = '%s/%s/%s' % (self._uri,
1481- self._series, revision)
1482- download_dir = path.join(project, self._series, revision)
1483- elif revision:
1484- revision_split = revision.split('/')
1485- if len(revision_split) != 2:
1486- raise EnvironmentError(
1487- 'Improper use of revision, needs to be formatted like'
1488- '[series]/[revision]. Use --list-revisions to find'
1489- 'the available revisions on cdimage')
1490- # TODO add verification that uri exists
1491- self._series = revision_split[0]
1492- download_uri = '%s/%s' % (self._uri, revision)
1493- download_dir = path.join(project, self._series, revision)
1494- else:
1495- uri = '%s/daily-preinstalled' % self._uri
1496- if pending:
1497- link = cdimage.get_latest_pending(uri)
1498- else:
1499- link = cdimage.get_latest_current(uri)
1500- download_uri = '%s/%s' % (uri, link)
1501- download_dir = path.join(project, link)
1502- self._download_uri = download_uri
1503- self._download_dir = download_dir
1504-
1505- def _load_hashes(self):
1506- self._hash_path = path.join(self._download_dir, self._hash_file_name)
1507- if path.exists(self._hash_path):
1508- with open(self._hash_path, 'r') as f:
1509- hash_content = f.read()
1510- else:
1511- hash_content = cdimage.get_sha256_content(
1512- '%s/%s' % (self._download_uri, self._hash_file_name))
1513- self._hash_content = hash_content
1514-
1515-
1516-class Environment(object):
1517- '''All the hacks to support multiple environments live here.'''
1518-
1519- @property
1520- def project(self):
1521- return self._project
1522-
1523- @property
1524- def download_dir(self):
1525- return self._download_dir
1526-
1527- @property
1528- def download_uri(self):
1529- return self._download_uri
1530-
1531- @property
1532- def bootstrap_files(self):
1533- return [self._files[i] for i in self._bootstrap_files]
1534-
1535- @property
1536- def recovery_files(self):
1537- return [self._files[i] for i in self._recovery_files]
1538-
1539- @property
1540- def device_zip_path(self):
1541- return self._files['device_zip']
1542-
1543- @property
1544- def ubuntu_zip_path(self):
1545- return self._files['ubuntu_zip']
1546-
1547- @property
1548- def system_img_path(self):
1549- return self._files['system_img']
1550-
1551- @property
1552- def boot_img_path(self):
1553- return self._files['boot_img']
1554-
1555- @property
1556- def recovery_img_path(self):
1557- return self._files['recovery_img']
1558-
1559- def __init__(self, preset_uri, device, series, latest_revision, revision,
1560- pending, legacy, base_path, settings):
1561- self._set_project(series, pending, legacy, base_path, latest_revision,
1562- revision, settings)
1563- self._set_download(preset_uri, settings)
1564- self._files, self._bootstrap_files, self._recovery_files = \
1565- self._set_files(device, settings)
1566-
1567- def _set_project(self, series, pending, legacy, base_path,
1568- latest_revision, revision, settings):
1569- if getattr(settings, 'revision', None):
1570- if not series:
1571- series = settings.default_series
1572- if legacy:
1573- project_name = 'ubuntu-touch-preview'
1574- else:
1575- project_name = 'ubuntu-touch'
1576- self._project = Project(
1577- project_name, series, settings.cdimage_uri_base, base_path,
1578- settings.hash_file_name, pending, revision, latest_revision)
1579- else:
1580- self._project = None
1581-
1582- def _set_download(self, preset_uri, settings):
1583- if preset_uri:
1584- self._download_uri = preset_uri
1585- elif not self._project:
1586- self._download_uri = settings.download_uri
1587- elif self._project:
1588- self._download_uri = self._project.download_uri
1589- else:
1590- raise EnvironmentError('Environment not setup correctly')
1591- if self._project:
1592- download_dir = path.join(settings.download_dir,
1593- self._project.download_dir)
1594- else:
1595- download_dir = path.join(settings.download_dir,
1596- get_ubuntu_stamp(self.download_uri))
1597- log.debug('Download set to %s' % self.download_uri)
1598- if self._download_uri:
1599- self._download_dir = get_download_dir_full_path(download_dir)
1600- else:
1601- self._download_dir = download_dir
1602-
1603- def _set_files(self, device, settings):
1604- files = {}
1605- project = self._project
1606- if project:
1607- templ = settings.files[project.name]
1608- files['ubuntu_zip'] = templ['ubuntu_zip'] % (project.series,)
1609- for key in templ:
1610- if key is 'ubuntu_zip':
1611- continue
1612- files[key] = templ[key] % (project.series, device)
1613- bootstrap_files = ['ubuntu_zip', 'system_img', 'boot_img',
1614- 'recovery_img']
1615- recovery_files = ['device_zip', 'ubuntu_zip']
1616- else:
1617- files['ubuntu_zip'] = None
1618- files['device_zip'] = None
1619- files['recovery_img'] = None
1620- files['system_img'] = settings.device_file_img % (device,)
1621- files['boot_img'] = settings.boot_file_img % (device,)
1622- bootstrap_files = ['system_img', 'boot_img']
1623- recovery_files = None
1624- for key in files:
1625- if not files[key]:
1626- continue
1627- files[key] = path.join(self._download_dir, files[key])
1628- return files, bootstrap_files, recovery_files
1629-
1630-
1631- def store_hashes(self):
1632- if not self._project.hash_content:
1633- return None
1634- with open(path.join(self._download_dir,
1635- self._project.hash_file_name), 'w') as f:
1636- for line in self._project.hash_content:
1637- f.write(line)
1638+def detect_device(serial, device=None):
1639+ '''If no argument passed determine them from the connected device.'''
1640+ # Check CyanogenMod property
1641+ if not device:
1642+ adb = AndroidBridge(serial)
1643+ adb.start()
1644+ device = adb.getprop('ro.cm.device').strip()
1645+ # Check Android property
1646+ if not device:
1647+ device = adb.getprop('ro.product.device').strip()
1648+ log.info('Device detected as %s' % device)
1649+ # property may not exist or we may not map it yet
1650+ if device not in settings.supported_devices:
1651+ raise EnvironmentError('Unsupported device, autodetect fails device')
1652+ return device
1653+
1654+
1655+#def setup_revision1(device, uri, download_dir, settings):
1656+# if not uri:
1657+# uri = settings.download_uri
1658+# if not download_dir:
1659+# build = get_ubuntu_stamp(uri)
1660+# download_dir = get_full_path(
1661+# os.path.join(settings.download_dir, build))
1662+# setup_download_directory(download_dir)
1663+# else:
1664+# uri = None
1665+# system = settings.device_file_img % device
1666+# boot = settings.boot_file_img % (device,)
1667+# hash_dict = {entry: load_hash(uri, '%s.md5sum' % entry, download_dir)
1668+# for entry in (system, boot)}
1669+# if uri:
1670+# system_uri = '%s/%s' % (uri, system),
1671+# boot_uri = '%s/%s' % (uri, boot),
1672+# else:
1673+# system_uri = None
1674+# boot_uri = None
1675+# system_file = resources.File(file_path=os.path.join(download_dir, system),
1676+# file_uri=system_uri,
1677+# file_hash=hashes[system],
1678+# file_hash_func=hashlib.md5)
1679+# boot_file = resources.File(file_path=os.path.join(download_dir, boot),
1680+# file_uri=boot_uri,
1681+# file_hash=hashes[boot],
1682+# file_hash_func=hashlib.md5)
1683+# return projects.Android(boot=boot_file, system=system_file)
1684+
1685+
1686+def setup_cdimage_files(project_name, uri, download_dir, series,
1687+ device, legacy=False):
1688+ downloads.setup_download_directory(download_dir)
1689+ templ_arch_any = settings.files_arch_any[project_name]
1690+ templ_arch_all = settings.files_arch_all[project_name]
1691+ file_names = {}
1692+ for key in templ_arch_any:
1693+ file_names[key] = templ_arch_any[key] % (series, device)
1694+ for key in templ_arch_all:
1695+ file_names[key] = templ_arch_all[key] % series
1696+ if legacy:
1697+ hash_func = hashlib.md5
1698+ hash_dict = {}
1699+ for key in file_names:
1700+ file_hash = hashes.load_hash(uri, '%s.md5sum' %
1701+ file_names[key], download_dir)
1702+ hash_dict[file_names[key]] = file_hash[file_names[key]]
1703+ else:
1704+ hash_func = hashlib.sha256
1705+ hash_dict = hashes.load_hash(uri, 'SHA256SUMS', download_dir)
1706+ files = {}
1707+ for key in file_names:
1708+ if uri:
1709+ file_uri = '%s/%s' % (uri, file_names[key])
1710+ else:
1711+ file_uri = None
1712+ files[key] = resources.File(
1713+ file_path=os.path.join(download_dir, file_names[key]),
1714+ file_uri=file_uri,
1715+ file_hash=hash_dict[file_names[key]],
1716+ file_hash_func=hash_func)
1717+ return files
1718+
1719+
1720+def setup_cdimage_touch(args):
1721+ device = detect_device(args.serial, args.device)
1722+ series = args.series
1723+ base_uri = '%s/%s' % (settings.cdimage_uri_base, args.project)
1724+
1725+ if args.base_path:
1726+ uri = None
1727+ download_dir = args.base_path
1728+
1729+ daily_uri = '%s/daily-preinstalled' % (base_uri, )
1730+ build = cdimage.get_build(daily_uri, args.pending)
1731+ uri = '%s/%s' % (daily_uri, build)
1732+ download_dir = downloads.get_full_path(
1733+ os.path.join(settings.download_dir, args.project, build))
1734+ files = setup_cdimage_files(
1735+ args.project, uri, download_dir, series, device)
1736+ return cdimage_project(files, args)
1737+
1738+
1739+def setup_cdimage_legacy(args):
1740+ series = args.series
1741+ uri = args.uri
1742+ device = detect_device(args.serial, args.device)
1743+ if args.base_path:
1744+ download_dir = args.base_path
1745+ elif args.revision or args.latest_revision:
1746+ build = args.build
1747+ uri = args.uri
1748+ download_dir = downloads.get_full_path(
1749+ os.path.join(args.project, series, build))
1750+ else:
1751+ base_uri = '%s/%s' % (settings.cdimage_uri_base, args.project)
1752+ daily_uri = '%s/daily-preinstalled' % (base_uri, )
1753+ build = cdimage.get_build(daily_uri)
1754+ uri = '%s/%s' % (daily_uri, build)
1755+ download_dir = downloads.get_full_path(
1756+ os.path.join(settings.download_dir, args.project, build))
1757+ files = setup_cdimage_files(
1758+ args.project, uri, download_dir, series, device, legacy=True)
1759+ return cdimage_project(files, args)
1760+
1761+
1762+def cdimage_project(files, args):
1763+ if args.bootstrap:
1764+ return projects.UbuntuTouchBootstrap(
1765+ system=files['system_img'],
1766+ boot=files['boot_img'],
1767+ recovery=files['recovery_img'],
1768+ ubuntu=files['ubuntu_zip'])
1769+ else:
1770+ if args.device_path:
1771+ files['device_zip'] = args.device_path
1772+ if args.ubuntu_path:
1773+ files['ubuntu_zip'] = args.ubuntu_path
1774+ return projects.UbuntuTouchRecovery(
1775+ device=files['device_zip'],
1776+ ubuntu=files['ubuntu_zip'],
1777+ wipe=args.wipe)
1778+
1779+
1780+def setup_ubuntu_system(args):
1781+ device = detect_device(args.serial, args.device)
1782+ if args.revision <= 0:
1783+ json = ubuntuimage.get_json_from_index(device, args.revision)
1784+ else:
1785+ raise EnvironmentError('Specific version retrieve not supported yet')
1786+ download_dir = downloads.get_full_path(os.path.join(
1787+ settings.download_dir, args.project, str(json['version'])))
1788+ uri = settings.system_image_uri
1789+ files, command_part = ubuntuimage.get_files(download_dir, uri, json)
1790+ recovery = cdimage.get_file(file_key='recovery_img',
1791+ series=args.series,
1792+ download_dir=download_dir,
1793+ device=device)
1794+ return projects.UbuntuTouchSystem(
1795+ file_list=files,
1796+ command_part=command_part,
1797+ recovery=recovery)
1798+
1799+
1800+def setup_community(args):
1801+ config_dir = community.branch_project(args.device)
1802+ json_dict = community.load_manifest(config_dir)
1803+ download_dir = community.get_download_dir(
1804+ args.device, json_dict['revision'])
1805+ files = community.get_files(json_dict, download_dir, args.series)
1806+ return projects.UbuntuTouchRecovery(
1807+ device=files['device'],
1808+ ubuntu=files['ubuntu'],
1809+ storage=json_dict['storage'],
1810+ wipe=args.wipe)
1811+
1812+
1813+def list_revisions(args):
1814+ # Easy hack to get rid of the logger and inhibit requests from logging
1815+ log.setLevel(logging.FATAL)
1816+ revisions = cdimage.get_available_revisions(args.uri)
1817+ cdimage.display_revisions(revisions)
1818
1819=== added file 'phabletutils/hashes.py'
1820--- phabletutils/hashes.py 1970-01-01 00:00:00 +0000
1821+++ phabletutils/hashes.py 2013-08-09 01:41:37 +0000
1822@@ -0,0 +1,66 @@
1823+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
1824+# Copyright (C) 2013 Canonical Ltd.
1825+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
1826+
1827+# This program is free software: you can redistribute it and/or modify
1828+# it under the terms of the GNU General Public License as published by
1829+# the Free Software Foundation; version 3 of the License.
1830+#
1831+# This program is distributed in the hope that it will be useful,
1832+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1833+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1834+# GNU General Public License for more details.
1835+#
1836+# You should have received a copy of the GNU General Public License
1837+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1838+
1839+import logging
1840+import os.path
1841+
1842+from phabletutils import downloads
1843+
1844+log = logging.getLogger()
1845+
1846+
1847+def hash2dict(hash_file_content):
1848+ '''Returns a dictionary with the sha256 sums for all files.'''
1849+ if not hash_file_content:
1850+ log.debug('hash file is empty')
1851+ return None
1852+ hash_list = filter((lambda x: len(x) is not 0),
1853+ hash_file_content.split('\n'))
1854+ hash_list = [h.split() for h in hash_list]
1855+ hash_dict = {}
1856+ for hash_entry in hash_list:
1857+ if hash_entry[1][0] == '*':
1858+ hash_entry[1] = hash_entry[1][1:]
1859+ hash_dict[hash_entry[1]] = hash_entry[0]
1860+ return hash_dict
1861+
1862+
1863+def load_hash(uri, artifact, download_dir=None):
1864+ if download_dir:
1865+ hash_path = os.path.join(download_dir, artifact)
1866+ else:
1867+ hash_path = None
1868+ hashes = {}
1869+ if hash_path and os.path.exists(hash_path):
1870+ with open(hash_path, 'r') as f:
1871+ file_hash_content = f.read()
1872+ hashes = hash2dict(file_hash_content)
1873+ if uri:
1874+ uri = '%s/%s' % (uri, artifact)
1875+ hash_content = downloads.get_content(uri)
1876+ if not hash_content:
1877+ raise RuntimeError('%s cannot be downloaded' % uri)
1878+ hashes.update(hash2dict(hash_content))
1879+ if hash_path:
1880+ log.debug('Storing hash to %s' % hash_path)
1881+ with downloads.flocked(hash_path):
1882+ with open(hash_path, 'w') as f:
1883+ for key in hashes:
1884+ f.write('%s %s\n' % (hashes[key], key))
1885+ if hashes:
1886+ return hashes
1887+ else:
1888+ raise RuntimeError('%s cannot be obtained for verifiaction' % artifact)
1889
1890=== added file 'phabletutils/license.py'
1891--- phabletutils/license.py 1970-01-01 00:00:00 +0000
1892+++ phabletutils/license.py 2013-08-09 01:41:37 +0000
1893@@ -0,0 +1,51 @@
1894+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
1895+# Copyright (C) 2013 Canonical Ltd.
1896+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
1897+
1898+# This program is free software: you can redistribute it and/or modify
1899+# it under the terms of the GNU General Public License as published by
1900+# the Free Software Foundation; version 3 of the License.
1901+#
1902+# This program is distributed in the hope that it will be useful,
1903+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1904+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1905+# GNU General Public License for more details.
1906+#
1907+# You should have received a copy of the GNU General Public License
1908+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1909+
1910+import os
1911+import logging
1912+
1913+log = logging.getLogger()
1914+
1915+
1916+def accepted(pathname):
1917+ '''
1918+ Remember that the user accepted the license.
1919+ '''
1920+ open(pathname, 'w').close()
1921+
1922+
1923+def has_accepted(pathname):
1924+ '''
1925+ Return True if the user accepted the license once.
1926+ '''
1927+ return os.path.exists(pathname)
1928+
1929+
1930+def query(message, accept_path):
1931+ '''Display end user agreement to continue with deployment.'''
1932+ try:
1933+ while True:
1934+ print message
1935+ print 'Do you accept? [yes|no]'
1936+ answer = raw_input().lower()
1937+ if answer == 'yes':
1938+ accepted(accept_path)
1939+ return True
1940+ elif answer == 'no':
1941+ return False
1942+ except KeyboardInterrupt:
1943+ log.error('Interruption detected, cancelling install')
1944+ return False
1945
1946=== added file 'phabletutils/projects.py'
1947--- phabletutils/projects.py 1970-01-01 00:00:00 +0000
1948+++ phabletutils/projects.py 2013-08-09 01:41:37 +0000
1949@@ -0,0 +1,237 @@
1950+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
1951+# Copyright (C) 2013 Canonical Ltd.
1952+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
1953+
1954+# This program is free software: you can redistribute it and/or modify
1955+# it under the terms of the GNU General Public License as published by
1956+# the Free Software Foundation; version 3 of the License.
1957+#
1958+# This program is distributed in the hope that it will be useful,
1959+# but WITHOUT ANY WARRANTY; without even the implied warranty of
1960+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1961+# GNU General Public License for more details.
1962+#
1963+# You should have received a copy of the GNU General Public License
1964+# along with this program. If not, see <http://www.gnu.org/licenses/>.
1965+
1966+"""Holds different projects or ways Ubuntu Touch is delivered."""
1967+
1968+import os
1969+import os.path
1970+import tempfile
1971+import logging
1972+import gzip
1973+
1974+from phabletutils.downloads import checksum_verify
1975+from phabletutils.resources import (File, SignedFile)
1976+from phabletutils import downloads
1977+from phabletutils import settings
1978+from time import sleep
1979+from textwrap import dedent
1980+
1981+log = logging.getLogger()
1982+
1983+
1984+def gunzip(file_path):
1985+ if not file_path.endswith('.gz'):
1986+ return file_path
1987+ target_path = file_path[:-3]
1988+ log.info('Decompressing %s' % file_path)
1989+ with open(target_path, 'wb') as target_file:
1990+ with gzip.open(file_path, 'r') as gzip_file:
1991+ for chunk in gzip_file.read():
1992+ target_file.write(chunk)
1993+ return target_path
1994+
1995+
1996+def wipe_device(adb):
1997+ log.info('Clearing /data and /cache')
1998+ adb.shell('mount /data')
1999+ adb.shell('rm -Rf /cache/* /data/* /data/.developer_mode')
2000+ adb.shell('mkdir /cache/recovery')
2001+ adb.shell('mkdir /data/media')
2002+
2003+
2004+class BaseProject(object):
2005+ """A provisioning mechanism for all the projects."""
2006+
2007+ def __init__(self, recovery=None, system=None, boot=None,
2008+ device=None, ubuntu=None, wipe=False):
2009+ self._list = []
2010+ for item in (recovery, system, boot, device, ubuntu):
2011+ if item and not isinstance(item, File):
2012+ raise TypeError('%s is not of type File' % item)
2013+ elif item:
2014+ self._list.append(item)
2015+ if item.check and not item.uri and not item.verified:
2016+ raise EnvironmentError(
2017+ '%s is not on disk' % item.path)
2018+
2019+ self._recovery = recovery
2020+ self._system = system
2021+ self._boot = boot
2022+ self._device = device
2023+ self._ubuntu = ubuntu
2024+ self._wipe = wipe
2025+
2026+ def download(self):
2027+ """Downloads and verifies resources."""
2028+ download_list = filter((lambda x: x.check), self._list)
2029+ download_list = filter((lambda x: not x.verified), download_list)
2030+ log.debug('Download list %s' % download_list)
2031+ if not download_list:
2032+ log.info('Download not required')
2033+ return
2034+ for entry in download_list:
2035+ log.debug('Download entry %s %s' % (entry.path, entry.verified))
2036+ downloads.download(entry)
2037+ if entry.hash and \
2038+ not checksum_verify(entry.path, entry.hash, entry.hash_type):
2039+ raise EnvironmentError(
2040+ 'Checksum does not match after download for %s '
2041+ 'and hash %s' % (entry.path, entry.hash))
2042+ download_list = filter(lambda x: isinstance(x, SignedFile), self._list)
2043+ for entry in download_list:
2044+ downloads.download_sig(entry)
2045+
2046+ def install(self):
2047+ raise EnvironmentError('Requires implementation')
2048+
2049+
2050+class Android(BaseProject):
2051+ """Standard Android Project."""
2052+
2053+ def __init__(self, boot, system):
2054+ super(Android, self).__init__(boot=boot, system=system)
2055+
2056+ def install(self, adb, fastboot):
2057+ log.warning('Device needs to be unlocked for the following to work')
2058+ fastboot.flash('system', gunzip(self.system))
2059+ fastboot.flash('boot', self.boot)
2060+ log.info('Installation will complete soon and reboot into Android')
2061+ fastboot.reboot()
2062+
2063+
2064+class UbuntuTouchBootstrap(BaseProject):
2065+
2066+ def __init__(self, boot, system, recovery, ubuntu):
2067+ log.debug('UbuntuTouchBootstrap '
2068+ 'boot: %s, system: %s, recovery: %s, ubuntu: %s' %
2069+ (boot.path, system.path, recovery.path, ubuntu.path))
2070+ super(UbuntuTouchBootstrap, self).__init__(
2071+ boot=boot, system=system, recovery=recovery, ubuntu=ubuntu,
2072+ wipe=True)
2073+
2074+ def install(self, adb, fastboot):
2075+ adb.reboot(bootloader=True)
2076+ log.warning('Device needs to be unlocked for the following to work')
2077+ fastboot.flash('system', self._system.path)
2078+ fastboot.flash('boot', self._boot.path)
2079+ fastboot.flash('recovery', self._recovery.path)
2080+ fastboot.boot(self._recovery.path)
2081+ sleep(15)
2082+ wipe_device(adb)
2083+ adb.push(self._ubuntu.path, '/sdcard/autodeploy.zip')
2084+ log.info('Deploying Ubuntu')
2085+ adb.reboot(recovery=True)
2086+ log.info('Installation will complete soon and reboot into Ubuntu')
2087+
2088+
2089+class UbuntuTouchRecovery(BaseProject):
2090+
2091+ recovery_script_template = dedent('''\
2092+ mount("{0}");
2093+ install_zip("{1}");
2094+ install_zip("{2}");
2095+ ''')
2096+
2097+ def __init__(self, device, ubuntu, storage='/sdcard/', wipe=False):
2098+ log.debug('UbuntuTouchRecovery device: %s, ubuntu: %s, wipe: %s' %
2099+ (device.path, ubuntu.path, wipe))
2100+ super(UbuntuTouchRecovery, self).__init__(
2101+ ubuntu=ubuntu, device=device, wipe=wipe)
2102+ self._storage = storage
2103+
2104+ def install(self, adb, fastboot=None):
2105+ """
2106+ Deploys recovery files, recovery script and then reboots to install.
2107+ """
2108+ log.warning('The device needs to have a clockwork mod recovery image '
2109+ '(or one that supports extendedcommands) '
2110+ 'in place for the provisioning to work')
2111+ adb.reboot(recovery=True)
2112+ sleep(20)
2113+ if self._wipe:
2114+ wipe_device(adb)
2115+ adb.shell('mount %s' % self._storage)
2116+ adb.push(self._device.path, self._storage)
2117+ adb.push(self._ubuntu.path, self._storage)
2118+ recovery_file = self.create_recovery_file()
2119+ adb.push(recovery_file, '/cache/recovery/extendedcommand')
2120+ adb.reboot(recovery=True)
2121+ log.info('Once completed the device should reboot into Ubuntu')
2122+ log.debug('Removing recovery file %s' % recovery_file)
2123+ os.unlink(recovery_file)
2124+
2125+ def create_recovery_file(self):
2126+ template = self.recovery_script_template
2127+ recovery_file = tempfile.NamedTemporaryFile(delete=False)
2128+ device = os.path.join(self._storage,
2129+ os.path.basename(self._device.path))
2130+ ubuntu = os.path.join(self._storage,
2131+ os.path.basename(self._ubuntu.path))
2132+ recovery_script = template.format(self._storage, device, ubuntu)
2133+ with recovery_file as output_file:
2134+ output_file.write(recovery_script)
2135+ return recovery_file.name
2136+
2137+
2138+class UbuntuTouchSystem(BaseProject):
2139+
2140+ ubuntu_recovery_script = dedent('''\
2141+ format data
2142+ format system
2143+ load_keyring image-master.tar.xz image-master.tar.xz.asc
2144+ load_keyring image-signing.tar.xz image-signing.tar.xz.asc
2145+ mount system
2146+ ''')
2147+
2148+ def __init__(self, file_list, recovery, command_part):
2149+ log.debug('UbuntuTouchSystem')
2150+ super(UbuntuTouchSystem, self).__init__(recovery=recovery, wipe=True)
2151+ for item in file_list:
2152+ if item and not isinstance(item, File):
2153+ raise TypeError('%s is not of type File' % item)
2154+ elif item:
2155+ self._list.append(item)
2156+ self._recovery_list = file_list
2157+ self._command_part = command_part
2158+
2159+ def install(self, adb, fastboot=None):
2160+ """
2161+ Deploys recovery files, recovery script and then reboots to install.
2162+ """
2163+ adb.reboot(recovery=True)
2164+ sleep(20)
2165+ wipe_device(adb)
2166+
2167+ for entry in self._recovery_list:
2168+ adb.push(entry.path, '/cache/recovery/')
2169+ try:
2170+ adb.push(entry.sig_path, '/cache/recovery/')
2171+ except AttributeError:
2172+ pass
2173+ adb.push(self.create_ubuntu_command_file(),
2174+ '/cache/recovery/ubuntu_command')
2175+ adb.reboot(bootloader=True)
2176+ fastboot.flash('recovery', self._recovery.path)
2177+ fastboot.boot(self._recovery.path)
2178+ log.info('Once completed the device should reboot into Ubuntu')
2179+
2180+ def create_ubuntu_command_file(self):
2181+ ubuntu_command_file = tempfile.NamedTemporaryFile(delete=False)
2182+ with ubuntu_command_file as output_file:
2183+ output_file.write(settings.ubuntu_recovery_script)
2184+ output_file.write(self._command_part)
2185+ output_file.write('unmount system\n')
2186+ return ubuntu_command_file.name
2187
2188=== added file 'phabletutils/resources.py'
2189--- phabletutils/resources.py 1970-01-01 00:00:00 +0000
2190+++ phabletutils/resources.py 2013-08-09 01:41:37 +0000
2191@@ -0,0 +1,82 @@
2192+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2193+# Copyright (C) 2013 Canonical Ltd.
2194+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
2195+
2196+# This program is free software: you can redistribute it and/or modify
2197+# it under the terms of the GNU General Public License as published by
2198+# the Free Software Foundation; version 3 of the License.
2199+#
2200+# This program is distributed in the hope that it will be useful,
2201+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2202+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2203+# GNU General Public License for more details.
2204+#
2205+# You should have received a copy of the GNU General Public License
2206+# along with this program. If not, see <http://www.gnu.org/licenses/>.
2207+
2208+"""Resources for downloading and installing Ubuntu Touch."""
2209+
2210+import downloads
2211+import hashlib
2212+import logging
2213+
2214+log = logging.getLogger()
2215+
2216+
2217+class File(object):
2218+ """A file object that can be downloaded and flashed."""
2219+
2220+ @property
2221+ def uri(self):
2222+ return self._uri
2223+
2224+ @property
2225+ def path(self):
2226+ return self._path
2227+
2228+ @property
2229+ def verified(self):
2230+ return self._verified
2231+
2232+ @property
2233+ def hash(self):
2234+ return self._hash
2235+
2236+ @property
2237+ def hash_type(self):
2238+ return self._hash_func
2239+
2240+ @property
2241+ def check(self):
2242+ return self._check
2243+
2244+ def __init__(self, file_uri, file_path, check=True, file_hash=None,
2245+ file_hash_func=hashlib.sha256):
2246+ self._uri = file_uri
2247+ self._path = file_path
2248+ self._hash = file_hash
2249+ self._hash_func = file_hash_func
2250+ self._check = check
2251+ if check and file_hash:
2252+ self._verified = downloads.checksum_verify(file_path, file_hash,
2253+ file_hash_func)
2254+ log.debug('%s verified: %s' % (file_path, self._verified))
2255+ else:
2256+ self._verified = False
2257+
2258+
2259+class SignedFile(File):
2260+
2261+ def sig_uri(self):
2262+ return self._sig_uri
2263+
2264+ @property
2265+ def sig_path(self):
2266+ return self._sig_path
2267+
2268+ def __init__(self, file_uri, file_path, file_hash, sig_path, sig_uri,
2269+ check=True, file_hash_func=hashlib.sha256):
2270+ super(SignedFile, self).__init__(
2271+ file_uri, file_path, check, file_hash, file_hash_func)
2272+ self._sig_path = sig_path
2273+ self.sig_uri = sig_uri
2274
2275=== modified file 'phabletutils/settings.py'
2276--- phabletutils/settings.py 2013-07-20 02:14:56 +0000
2277+++ phabletutils/settings.py 2013-08-09 01:41:37 +0000
2278@@ -1,18 +1,18 @@
2279-#! /usr/bin/env python
2280-# This program is free software: you can redistribute it and/or modify it
2281-# under the terms of the the GNU General Public License version 3, as
2282-# published by the Free Software Foundation.
2283-#
2284-# This program is distributed in the hope that it will be useful, but
2285-# WITHOUT ANY WARRANTY; without even the implied warranties of
2286-# MERCHANTABILITY, SATISFACTORY QUALITY or FITNESS FOR A PARTICULAR
2287-# PURPOSE. See the applicable version of the GNU Lesser General Public
2288-# License for more details.
2289-#.
2290+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2291+# Copyright (C) 2013 Canonical Ltd.
2292+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
2293+
2294+# This program is free software: you can redistribute it and/or modify
2295+# it under the terms of the GNU General Public License as published by
2296+# the Free Software Foundation; version 3 of the License.
2297+#
2298+# This program is distributed in the hope that it will be useful,
2299+# but WITHOUT ANY WARRANTY; without even the implied warranty of
2300+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2301+# GNU General Public License for more details.
2302+#
2303 # You should have received a copy of the GNU General Public License
2304 # along with this program. If not, see <http://www.gnu.org/licenses/>.
2305-#
2306-# Copyright (C) 2013 Canonical, Ltd.
2307
2308 revision = 2
2309 default_series = 'saucy'
2310@@ -20,16 +20,14 @@
2311 system_image_uri = 'https://system-image.ubuntu.com'
2312 download_dir = 'phablet-flash'
2313
2314-files = {
2315+files_arch_any = {
2316 'ubuntu-touch': {
2317- 'ubuntu_zip': '%s-preinstalled-touch-armhf.zip',
2318 'device_zip': '%s-preinstalled-touch-armel+%s.zip',
2319 'system_img': '%s-preinstalled-system-armel+%s.img',
2320 'boot_img': '%s-preinstalled-boot-armhf+%s.img',
2321 'recovery_img': '%s-preinstalled-recovery-armel+%s.img',
2322 },
2323 'ubuntu-touch-preview': {
2324- 'ubuntu_zip': '%s-preinstalled-phablet-armhf.zip',
2325 'device_zip': '%s-preinstalled-armel+%s.zip',
2326 'system_img': '%s-preinstalled-system-armel+%s.img',
2327 'boot_img': '%s-preinstalled-boot-armel+%s.img',
2328@@ -37,20 +35,21 @@
2329 },
2330 }
2331
2332+files_arch_all = {
2333+ 'ubuntu-touch': {
2334+ 'ubuntu_zip': '%s-preinstalled-touch-armhf.zip',
2335+ },
2336+ 'ubuntu-touch-preview': {
2337+ 'ubuntu_zip': '%s-preinstalled-phablet-armhf.zip',
2338+ }
2339+}
2340+
2341 recovery_script_template = '''boot-recovery
2342---update_package={0}/{1}
2343---user_data_update_package={0}/{2}
2344+--update_package=/sdcard/{0}
2345+--user_data_update_package=/sdcard/{1}
2346 reboot
2347 '''
2348
2349-ubuntu_recovery_script = '''format data
2350-format system
2351-load_keyring image-master.tar.xz image-master.tar.xz.asc
2352-load_keyring image-signing.tar.xz image-signing.tar.xz.asc
2353-mount system
2354-'''
2355-hash_file_name = 'SHA256SUMS'
2356-
2357 supported_devices = ('mako',
2358 'maguro',
2359 'manta',
2360
2361=== modified file 'phabletutils/ubuntuimage.py'
2362--- phabletutils/ubuntuimage.py 2013-08-02 22:01:03 +0000
2363+++ phabletutils/ubuntuimage.py 2013-08-09 01:41:37 +0000
2364@@ -1,3 +1,4 @@
2365+# -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
2366 # Copyright (C) 2013 Canonical Ltd.
2367 # Author: Sergio Schvezov <sergio.schvezov@canonical.com>
2368
2369@@ -15,50 +16,50 @@
2370
2371 import json
2372 import os.path
2373-import requests
2374
2375 from phabletutils import downloads
2376+from phabletutils import resources
2377 from phabletutils import settings
2378
2379
2380-def get_json_from_index(device, index=-1):
2381+def get_json_from_index(device, index):
2382 """Returns json index for device"""
2383+ index -= 1
2384 json_index_uri = '%s/daily/%s/index.json' % \
2385 (settings.system_image_uri, device)
2386- json_index_request = requests.get(json_index_uri)
2387- json_index = json.loads(json_index_request.content)
2388- version_data = sorted([entry for entry in json_index['images']
2389- if entry['type'] == "full"],
2390- key=lambda entry: entry['version'])[index]
2391- return version_data
2392-
2393-
2394-def download_images(download_dir, json_list):
2395+ json_content = downloads.get_content(json_index_uri)
2396+ if not json_content:
2397+ raise RuntimeError('%s cannot be retrieved' % json_index_uri)
2398+ json_index = json.loads(json_content)
2399+ json_dict = sorted([entry for entry in json_index['images']
2400+ if entry['type'] == "full"],
2401+ key=lambda entry: entry['version'])[index]
2402+ return json_dict
2403+
2404+
2405+def get_files(download_dir, uri, json):
2406 files = {}
2407- files['updates'] = []
2408- for entry in sorted(json_list['files'], key=lambda entry: entry['order']):
2409+ command_part = ''
2410+ files = []
2411+ for entry in sorted(json['files'], key=lambda entry: entry['order']):
2412 filename = entry['path'].split("/")[-1]
2413 signame = entry['signature'].split("/")[-1]
2414- filename_path = os.path.join(download_dir, filename)
2415- signame_path = os.path.join(download_dir, signame)
2416- filename_uri = '%s/%s' % (settings.system_image_uri, entry['path'])
2417- signame_uri = '%s/%s' % (settings.system_image_uri, entry['signature'])
2418- with downloads.flocked(filename_path):
2419- downloads.download(filename_uri, filename_path)
2420- downloads.download(signame_uri, signame_path)
2421- files['updates'].append({'filename': filename_path,
2422- 'signame': signame_path})
2423- files['base'] = []
2424+ f = resources.SignedFile(
2425+ file_path=os.path.join(download_dir, filename),
2426+ sig_path=os.path.join(download_dir, signame),
2427+ file_uri='%s/%s' % (uri, entry['path']),
2428+ sig_uri='%s/%s' % (uri, entry['signature']),
2429+ file_hash=entry['checksum'])
2430+ files.append(f)
2431+ command_part += 'update %s %s\n' % (filename, signame)
2432 for keyring in ('image-master', 'image-signing'):
2433 filename = '%s.tar.xz' % keyring
2434 signame = '%s.asc' % filename
2435- filename_path = os.path.join(download_dir, filename)
2436- signame_path = os.path.join(download_dir, signame)
2437- filename_uri = '%s/gpg/%s' % (settings.system_image_uri, filename)
2438- signame_uri = '%s.asc' % filename_uri
2439- with downloads.flocked(filename_path):
2440- downloads.download(filename_uri, filename_path)
2441- downloads.download(signame_uri, signame_path)
2442- files['base'].append({'filename': filename_path,
2443- 'signame': signame_path})
2444- return files
2445+ f = resources.SignedFile(
2446+ file_path=os.path.join(download_dir, filename),
2447+ sig_path=os.path.join(download_dir, signame),
2448+ file_uri='%s/gpg/%s' % (uri, filename),
2449+ sig_uri='%s/gpg/%s.asc' % (uri, filename),
2450+ file_hash=None)
2451+ files.append(f)
2452+ return files, command_part
2453
2454=== added file 'tests/index.json'
2455--- tests/index.json 1970-01-01 00:00:00 +0000
2456+++ tests/index.json 2013-08-09 01:41:37 +0000
2457@@ -0,0 +1,577 @@
2458+{
2459+ "global": {
2460+ "generated_at": "Wed Aug 07 11:30:01 UTC 2013"
2461+ },
2462+ "images": [
2463+ {
2464+ "description": "20130731.3",
2465+ "files": [
2466+ {
2467+ "checksum": "7c9ce28bb751e25e80e5cee540ef1bdc929e97801528a14f066516f3ae2343d1",
2468+ "order": 0,
2469+ "path": "/daily/ubuntu/ubuntu-20130731.3.full.tar.xz",
2470+ "signature": "/daily/ubuntu/ubuntu-20130731.3.full.tar.xz.asc",
2471+ "size": 264081876
2472+ },
2473+ {
2474+ "checksum": "22ee6ded6902004e341a83c284218510a2b3a92c537a86140aa0daa76b7852f4",
2475+ "order": 1,
2476+ "path": "/daily/mako/mako-20130731.3.full.tar.xz",
2477+ "signature": "/daily/mako/mako-20130731.3.full.tar.xz.asc",
2478+ "size": 37335764
2479+ },
2480+ {
2481+ "checksum": "4181ff7e4f6f6262f5eb231e65735c277abf5a13993d68abb200ca9cd8136c4e",
2482+ "order": 2,
2483+ "path": "/daily/mako/version-20130733.tar.xz",
2484+ "signature": "/daily/mako/version-20130733.tar.xz.asc",
2485+ "size": 196
2486+ }
2487+ ],
2488+ "type": "full",
2489+ "version": 20130733
2490+ },
2491+ {
2492+ "base": 20130732,
2493+ "description": "20130731.3",
2494+ "files": [
2495+ {
2496+ "checksum": "002c10476be7a4db9d2fda3b9583eb67c2873b82c5f95483f6df7b253d7a8171",
2497+ "order": 0,
2498+ "path": "/daily/ubuntu/ubuntu-20130731.3.delta-20130731.tar.xz",
2499+ "signature": "/daily/ubuntu/ubuntu-20130731.3.delta-20130731.tar.xz.asc",
2500+ "size": 14993132
2501+ },
2502+ {
2503+ "checksum": "e557a50f29ecfe6265c405cbc35621f18e2b01225f9d9bfea01e30ddca752f43",
2504+ "order": 1,
2505+ "path": "/daily/mako/mako-20130731.3.delta-20130730.tar.xz",
2506+ "signature": "/daily/mako/mako-20130731.3.delta-20130730.tar.xz.asc",
2507+ "size": 34903260
2508+ },
2509+ {
2510+ "checksum": "4181ff7e4f6f6262f5eb231e65735c277abf5a13993d68abb200ca9cd8136c4e",
2511+ "order": 2,
2512+ "path": "/daily/mako/version-20130733.tar.xz",
2513+ "signature": "/daily/mako/version-20130733.tar.xz.asc",
2514+ "size": 196
2515+ }
2516+ ],
2517+ "type": "delta",
2518+ "version": 20130733
2519+ },
2520+ {
2521+ "description": "20130801",
2522+ "files": [
2523+ {
2524+ "checksum": "963c3bf9367450ab17c47f670f91db1d15f0ef6bdcf194481b5f458379129d7e",
2525+ "order": 0,
2526+ "path": "/daily/ubuntu/ubuntu-20130801.full.tar.xz",
2527+ "signature": "/daily/ubuntu/ubuntu-20130801.full.tar.xz.asc",
2528+ "size": 264000016
2529+ },
2530+ {
2531+ "checksum": "eb2160d8e37b9e5af34a8cb8b9e6cc63105f759a953d928c896741feef7eed01",
2532+ "order": 1,
2533+ "path": "/daily/mako/mako-20130801.full.tar.xz",
2534+ "signature": "/daily/mako/mako-20130801.full.tar.xz.asc",
2535+ "size": 37335924
2536+ },
2537+ {
2538+ "checksum": "11da343b235e14a10a2537620e0dc397f4a23e8d5fc6f81e7fa86dcc9e80fa6a",
2539+ "order": 2,
2540+ "path": "/daily/mako/version-20130800.tar.xz",
2541+ "signature": "/daily/mako/version-20130800.tar.xz.asc",
2542+ "size": 192
2543+ }
2544+ ],
2545+ "type": "full",
2546+ "version": 20130800
2547+ },
2548+ {
2549+ "base": 20130733,
2550+ "description": "20130801",
2551+ "files": [
2552+ {
2553+ "checksum": "f81a98769b92a9ba2c61b75ee83aa7fa7417767788f518553ae195e0a846768f",
2554+ "order": 0,
2555+ "path": "/daily/ubuntu/ubuntu-20130801.delta-20130731.3.tar.xz",
2556+ "signature": "/daily/ubuntu/ubuntu-20130801.delta-20130731.3.tar.xz.asc",
2557+ "size": 18625000
2558+ },
2559+ {
2560+ "checksum": "e907a5a4740881080bb997e494df69e4b317ae938a2a689d0aa5cf7a086ea6a4",
2561+ "order": 1,
2562+ "path": "/daily/mako/mako-20130801.delta-20130731.3.tar.xz",
2563+ "signature": "/daily/mako/mako-20130801.delta-20130731.3.tar.xz.asc",
2564+ "size": 34903248
2565+ },
2566+ {
2567+ "checksum": "11da343b235e14a10a2537620e0dc397f4a23e8d5fc6f81e7fa86dcc9e80fa6a",
2568+ "order": 2,
2569+ "path": "/daily/mako/version-20130800.tar.xz",
2570+ "signature": "/daily/mako/version-20130800.tar.xz.asc",
2571+ "size": 192
2572+ }
2573+ ],
2574+ "type": "delta",
2575+ "version": 20130800
2576+ },
2577+ {
2578+ "description": "20130802.1",
2579+ "files": [
2580+ {
2581+ "checksum": "05969428a0502d021ee91945f28bfd9884f774f406fe7e0578b09e26f521b171",
2582+ "order": 0,
2583+ "path": "/daily/ubuntu/ubuntu-20130802.1.full.tar.xz",
2584+ "signature": "/daily/ubuntu/ubuntu-20130802.1.full.tar.xz.asc",
2585+ "size": 264078540
2586+ },
2587+ {
2588+ "checksum": "269a33c87c6a995f068293776aa10f4a74d0cbb8101a236e0fedb375aff1df63",
2589+ "order": 1,
2590+ "path": "/daily/mako/mako-20130802.1.full.tar.xz",
2591+ "signature": "/daily/mako/mako-20130802.1.full.tar.xz.asc",
2592+ "size": 37342212
2593+ },
2594+ {
2595+ "checksum": "b2f4f1b04add4d47209598acb78137e3eb31ef1997aa92f67b17fcba8ea2911c",
2596+ "order": 2,
2597+ "path": "/daily/mako/version-20130801.tar.xz",
2598+ "signature": "/daily/mako/version-20130801.tar.xz.asc",
2599+ "size": 192
2600+ }
2601+ ],
2602+ "type": "full",
2603+ "version": 20130801
2604+ },
2605+ {
2606+ "base": 20130800,
2607+ "description": "20130802.1",
2608+ "files": [
2609+ {
2610+ "checksum": "d3c80a6144159933d037a605052b1c56ad4edae05ba31c73841c4bc80df59f6e",
2611+ "order": 0,
2612+ "path": "/daily/ubuntu/ubuntu-20130802.1.delta-20130801.tar.xz",
2613+ "signature": "/daily/ubuntu/ubuntu-20130802.1.delta-20130801.tar.xz.asc",
2614+ "size": 18381112
2615+ },
2616+ {
2617+ "checksum": "0fe99c8d57cda68dc757bb10d039f26895c822d7e5d6efb96f9456efd65c384f",
2618+ "order": 1,
2619+ "path": "/daily/mako/mako-20130802.1.delta-20130801.tar.xz",
2620+ "signature": "/daily/mako/mako-20130802.1.delta-20130801.tar.xz.asc",
2621+ "size": 37098756
2622+ },
2623+ {
2624+ "checksum": "b2f4f1b04add4d47209598acb78137e3eb31ef1997aa92f67b17fcba8ea2911c",
2625+ "order": 2,
2626+ "path": "/daily/mako/version-20130801.tar.xz",
2627+ "signature": "/daily/mako/version-20130801.tar.xz.asc",
2628+ "size": 192
2629+ }
2630+ ],
2631+ "type": "delta",
2632+ "version": 20130801
2633+ },
2634+ {
2635+ "description": "20130802.2",
2636+ "files": [
2637+ {
2638+ "checksum": "253139a135389d920ce446d258e4b0c779c51780c1ad747641a6cb11a3331634",
2639+ "order": 0,
2640+ "path": "/daily/ubuntu/ubuntu-20130802.2.full.tar.xz",
2641+ "signature": "/daily/ubuntu/ubuntu-20130802.2.full.tar.xz.asc",
2642+ "size": 264050500
2643+ },
2644+ {
2645+ "checksum": "490fc1586ea46d98719f2d53ae3044c31ef065d7d353071aed9ca4af43e37ceb",
2646+ "order": 1,
2647+ "path": "/daily/mako/mako-20130802.2.full.tar.xz",
2648+ "signature": "/daily/mako/mako-20130802.2.full.tar.xz.asc",
2649+ "size": 37335972
2650+ },
2651+ {
2652+ "checksum": "64fbb613532c31d2ee0d6141a2cba656c3247f39a257b8eed67fa1d18f481983",
2653+ "order": 2,
2654+ "path": "/daily/mako/version-20130802.tar.xz",
2655+ "signature": "/daily/mako/version-20130802.tar.xz.asc",
2656+ "size": 192
2657+ }
2658+ ],
2659+ "type": "full",
2660+ "version": 20130802
2661+ },
2662+ {
2663+ "base": 20130801,
2664+ "description": "20130802.2",
2665+ "files": [
2666+ {
2667+ "checksum": "193fde8008c63009397cb5d470c0d81c3547583ea5a28e1c1dc0c7be57ab59cc",
2668+ "order": 0,
2669+ "path": "/daily/ubuntu/ubuntu-20130802.2.delta-20130802.1.tar.xz",
2670+ "signature": "/daily/ubuntu/ubuntu-20130802.2.delta-20130802.1.tar.xz.asc",
2671+ "size": 19171116
2672+ },
2673+ {
2674+ "checksum": "8f3747431d40a45e52c780116f92939ac4cd071d63a80d0c9031b1da6b6241ac",
2675+ "order": 1,
2676+ "path": "/daily/mako/mako-20130802.2.delta-20130802.1.tar.xz",
2677+ "signature": "/daily/mako/mako-20130802.2.delta-20130802.1.tar.xz.asc",
2678+ "size": 34996120
2679+ },
2680+ {
2681+ "checksum": "64fbb613532c31d2ee0d6141a2cba656c3247f39a257b8eed67fa1d18f481983",
2682+ "order": 2,
2683+ "path": "/daily/mako/version-20130802.tar.xz",
2684+ "signature": "/daily/mako/version-20130802.tar.xz.asc",
2685+ "size": 192
2686+ }
2687+ ],
2688+ "type": "delta",
2689+ "version": 20130802
2690+ },
2691+ {
2692+ "description": "20130803",
2693+ "files": [
2694+ {
2695+ "checksum": "c0bc116b31adcb88b4587f8db0af9dcc6c6a2be839ea6013c5bd970bf9625467",
2696+ "order": 0,
2697+ "path": "/daily/ubuntu/ubuntu-20130803.full.tar.xz",
2698+ "signature": "/daily/ubuntu/ubuntu-20130803.full.tar.xz.asc",
2699+ "size": 264111912
2700+ },
2701+ {
2702+ "checksum": "02730cefdb71d35ce971f9494b0fedb775e1fd08cb65a8826fd74b3ebdba55d0",
2703+ "order": 1,
2704+ "path": "/daily/mako/mako-20130803.full.tar.xz",
2705+ "signature": "/daily/mako/mako-20130803.full.tar.xz.asc",
2706+ "size": 37344168
2707+ },
2708+ {
2709+ "checksum": "3b4f2fdce15f5f3ececdc73b2b58f512717da37abadd50d9ac08fff4dcafad00",
2710+ "order": 2,
2711+ "path": "/daily/mako/version-20130803.tar.xz",
2712+ "signature": "/daily/mako/version-20130803.tar.xz.asc",
2713+ "size": 196
2714+ }
2715+ ],
2716+ "type": "full",
2717+ "version": 20130803
2718+ },
2719+ {
2720+ "base": 20130802,
2721+ "description": "20130803",
2722+ "files": [
2723+ {
2724+ "checksum": "fd11b3944464bde87f7d187979eef556ce33a3f5f6fff00808bffeaea906b0ee",
2725+ "order": 0,
2726+ "path": "/daily/ubuntu/ubuntu-20130803.delta-20130802.2.tar.xz",
2727+ "signature": "/daily/ubuntu/ubuntu-20130803.delta-20130802.2.tar.xz.asc",
2728+ "size": 14519416
2729+ },
2730+ {
2731+ "checksum": "60f988ece3a4a092e0f05aa326feb655d67b68e2a53736bd4aa6717a3e0d26db",
2732+ "order": 1,
2733+ "path": "/daily/mako/mako-20130803.delta-20130802.2.tar.xz",
2734+ "signature": "/daily/mako/mako-20130803.delta-20130802.2.tar.xz.asc",
2735+ "size": 34918944
2736+ },
2737+ {
2738+ "checksum": "3b4f2fdce15f5f3ececdc73b2b58f512717da37abadd50d9ac08fff4dcafad00",
2739+ "order": 2,
2740+ "path": "/daily/mako/version-20130803.tar.xz",
2741+ "signature": "/daily/mako/version-20130803.tar.xz.asc",
2742+ "size": 196
2743+ }
2744+ ],
2745+ "type": "delta",
2746+ "version": 20130803
2747+ },
2748+ {
2749+ "description": "20130804",
2750+ "files": [
2751+ {
2752+ "checksum": "f3c1273666c71cb3fd74d9762783c016794cf36f5aab3fa01329460966293c6c",
2753+ "order": 0,
2754+ "path": "/daily/ubuntu/ubuntu-20130804.full.tar.xz",
2755+ "signature": "/daily/ubuntu/ubuntu-20130804.full.tar.xz.asc",
2756+ "size": 264089004
2757+ },
2758+ {
2759+ "checksum": "5f49821d302631c5de2f5df838b168b2c8290c0a9a4a510533464d92e4b9b638",
2760+ "order": 1,
2761+ "path": "/daily/mako/mako-20130804.full.tar.xz",
2762+ "signature": "/daily/mako/mako-20130804.full.tar.xz.asc",
2763+ "size": 37335780
2764+ },
2765+ {
2766+ "checksum": "e2619bd73b47539e23ebfbc29620baf8a6d56c3cf850141e2f9b3c8973e00342",
2767+ "order": 2,
2768+ "path": "/daily/mako/version-20130804.tar.xz",
2769+ "signature": "/daily/mako/version-20130804.tar.xz.asc",
2770+ "size": 196
2771+ }
2772+ ],
2773+ "type": "full",
2774+ "version": 20130804
2775+ },
2776+ {
2777+ "base": 20130803,
2778+ "description": "20130804",
2779+ "files": [
2780+ {
2781+ "checksum": "80c02eb1269adfd3b641593a2a6c93a4f4c570fbce1a90020073295276a1c0ba",
2782+ "order": 0,
2783+ "path": "/daily/ubuntu/ubuntu-20130804.delta-20130803.tar.xz",
2784+ "signature": "/daily/ubuntu/ubuntu-20130804.delta-20130803.tar.xz.asc",
2785+ "size": 16620436
2786+ },
2787+ {
2788+ "checksum": "99d0eb9f2315675a427c899ed308ab2ce11694675c979a305800f1949e008083",
2789+ "order": 1,
2790+ "path": "/daily/mako/mako-20130804.delta-20130803.tar.xz",
2791+ "signature": "/daily/mako/mako-20130804.delta-20130803.tar.xz.asc",
2792+ "size": 34913316
2793+ },
2794+ {
2795+ "checksum": "e2619bd73b47539e23ebfbc29620baf8a6d56c3cf850141e2f9b3c8973e00342",
2796+ "order": 2,
2797+ "path": "/daily/mako/version-20130804.tar.xz",
2798+ "signature": "/daily/mako/version-20130804.tar.xz.asc",
2799+ "size": 196
2800+ }
2801+ ],
2802+ "type": "delta",
2803+ "version": 20130804
2804+ },
2805+ {
2806+ "description": "20130805",
2807+ "files": [
2808+ {
2809+ "checksum": "6862856f1e029fd86883e1b28cc4aa959190ff3fdcaeb3c4115184c2942d21ee",
2810+ "order": 0,
2811+ "path": "/daily/ubuntu/ubuntu-20130805.full.tar.xz",
2812+ "signature": "/daily/ubuntu/ubuntu-20130805.full.tar.xz.asc",
2813+ "size": 264023912
2814+ },
2815+ {
2816+ "checksum": "c9f9b8644d5573cd73727a9a506ca46663cf22b6bae82d0dad90f3e53f43079c",
2817+ "order": 1,
2818+ "path": "/daily/mako/mako-20130805.full.tar.xz",
2819+ "signature": "/daily/mako/mako-20130805.full.tar.xz.asc",
2820+ "size": 37343256
2821+ },
2822+ {
2823+ "checksum": "915f703e637a509c9947890983427ec6d44fa53ee5a45e866acbd76c5c7d2fcb",
2824+ "order": 2,
2825+ "path": "/daily/mako/version-20130805.tar.xz",
2826+ "signature": "/daily/mako/version-20130805.tar.xz.asc",
2827+ "size": 196
2828+ }
2829+ ],
2830+ "type": "full",
2831+ "version": 20130805
2832+ },
2833+ {
2834+ "base": 20130804,
2835+ "description": "20130805",
2836+ "files": [
2837+ {
2838+ "checksum": "8d65c5deb8723db8df7e918790f98bfe4304490d7bde96af85fba0989ff04fe8",
2839+ "order": 0,
2840+ "path": "/daily/ubuntu/ubuntu-20130805.delta-20130804.tar.xz",
2841+ "signature": "/daily/ubuntu/ubuntu-20130805.delta-20130804.tar.xz.asc",
2842+ "size": 15166336
2843+ },
2844+ {
2845+ "checksum": "401400168374635c26c320218518565d567bd3c38d0a4ffc2ff750207702d070",
2846+ "order": 1,
2847+ "path": "/daily/mako/mako-20130805.delta-20130804.tar.xz",
2848+ "signature": "/daily/mako/mako-20130805.delta-20130804.tar.xz.asc",
2849+ "size": 34921568
2850+ },
2851+ {
2852+ "checksum": "915f703e637a509c9947890983427ec6d44fa53ee5a45e866acbd76c5c7d2fcb",
2853+ "order": 2,
2854+ "path": "/daily/mako/version-20130805.tar.xz",
2855+ "signature": "/daily/mako/version-20130805.tar.xz.asc",
2856+ "size": 196
2857+ }
2858+ ],
2859+ "type": "delta",
2860+ "version": 20130805
2861+ },
2862+ {
2863+ "description": "20130806",
2864+ "files": [
2865+ {
2866+ "checksum": "cd06ae29bc3d57def4d216be24d857a0c01367f65626651459a9478c42a8d80c",
2867+ "order": 0,
2868+ "path": "/daily/ubuntu/ubuntu-20130806.full.tar.xz",
2869+ "signature": "/daily/ubuntu/ubuntu-20130806.full.tar.xz.asc",
2870+ "size": 264142320
2871+ },
2872+ {
2873+ "checksum": "db5a43ee320eac7474a98beb0bbcc43648ff17f90adf581d19200bfb5bda0dce",
2874+ "order": 1,
2875+ "path": "/daily/mako/mako-20130806.full.tar.xz",
2876+ "signature": "/daily/mako/mako-20130806.full.tar.xz.asc",
2877+ "size": 42475196
2878+ },
2879+ {
2880+ "checksum": "3a17c0ae8f9a42c26aee06313ebc619f0b2a24385f7cbf3df519698ab9078acf",
2881+ "order": 2,
2882+ "path": "/daily/mako/version-20130806.tar.xz",
2883+ "signature": "/daily/mako/version-20130806.tar.xz.asc",
2884+ "size": 196
2885+ }
2886+ ],
2887+ "type": "full",
2888+ "version": 20130806
2889+ },
2890+ {
2891+ "base": 20130805,
2892+ "description": "20130806",
2893+ "files": [
2894+ {
2895+ "checksum": "0acc8ef06a657d74117a759b7e0fff0ba7a69e332aec15f8dbc7d1716d04fbbb",
2896+ "order": 0,
2897+ "path": "/daily/ubuntu/ubuntu-20130806.delta-20130805.tar.xz",
2898+ "signature": "/daily/ubuntu/ubuntu-20130806.delta-20130805.tar.xz.asc",
2899+ "size": 17806408
2900+ },
2901+ {
2902+ "checksum": "59d1604655efc0cfc582b97bae1111344f273b49c3df259647fe6fb063448ebb",
2903+ "order": 1,
2904+ "path": "/daily/mako/mako-20130806.delta-20130805.tar.xz",
2905+ "signature": "/daily/mako/mako-20130806.delta-20130805.tar.xz.asc",
2906+ "size": 42221272
2907+ },
2908+ {
2909+ "checksum": "3a17c0ae8f9a42c26aee06313ebc619f0b2a24385f7cbf3df519698ab9078acf",
2910+ "order": 2,
2911+ "path": "/daily/mako/version-20130806.tar.xz",
2912+ "signature": "/daily/mako/version-20130806.tar.xz.asc",
2913+ "size": 196
2914+ }
2915+ ],
2916+ "type": "delta",
2917+ "version": 20130806
2918+ },
2919+ {
2920+ "description": "20130806.1",
2921+ "files": [
2922+ {
2923+ "checksum": "b42156ae0b171c3425cb832d46cc4a86302a943beea18a29132ba3f84d978a02",
2924+ "order": 0,
2925+ "path": "/daily/ubuntu/ubuntu-20130806.1.full.tar.xz",
2926+ "signature": "/daily/ubuntu/ubuntu-20130806.1.full.tar.xz.asc",
2927+ "size": 264114544
2928+ },
2929+ {
2930+ "checksum": "6b5f8a78e94554ad59a3bb7f81550cce446d17d25d455759a77c6435d70de715",
2931+ "order": 1,
2932+ "path": "/daily/mako/mako-20130806.1.full.tar.xz",
2933+ "signature": "/daily/mako/mako-20130806.1.full.tar.xz.asc",
2934+ "size": 36869564
2935+ },
2936+ {
2937+ "checksum": "aa1fbad7150d4c0a2cc23e78672678aaeeaa26ac4e57a65fb635bf62c898955c",
2938+ "order": 2,
2939+ "path": "/daily/mako/version-20130807.tar.xz",
2940+ "signature": "/daily/mako/version-20130807.tar.xz.asc",
2941+ "size": 188
2942+ }
2943+ ],
2944+ "type": "full",
2945+ "version": 20130807
2946+ },
2947+ {
2948+ "base": 20130806,
2949+ "description": "20130806.1",
2950+ "files": [
2951+ {
2952+ "checksum": "3d6283525d795a34f98e2b996ae5bf7ce70892ca79026c776fca973b1080b30e",
2953+ "order": 0,
2954+ "path": "/daily/ubuntu/ubuntu-20130806.1.delta-20130806.tar.xz",
2955+ "signature": "/daily/ubuntu/ubuntu-20130806.1.delta-20130806.tar.xz.asc",
2956+ "size": 14764572
2957+ },
2958+ {
2959+ "checksum": "3e02f4a2ae19f47ba1bd10c2f236b46b368d012ede8003cb11fa3afbe0d7683f",
2960+ "order": 1,
2961+ "path": "/daily/mako/mako-20130806.1.delta-20130806.tar.xz",
2962+ "signature": "/daily/mako/mako-20130806.1.delta-20130806.tar.xz.asc",
2963+ "size": 34428224
2964+ },
2965+ {
2966+ "checksum": "aa1fbad7150d4c0a2cc23e78672678aaeeaa26ac4e57a65fb635bf62c898955c",
2967+ "order": 2,
2968+ "path": "/daily/mako/version-20130807.tar.xz",
2969+ "signature": "/daily/mako/version-20130807.tar.xz.asc",
2970+ "size": 188
2971+ }
2972+ ],
2973+ "type": "delta",
2974+ "version": 20130807
2975+ },
2976+ {
2977+ "description": "20130807",
2978+ "files": [
2979+ {
2980+ "checksum": "5a615f41c92c562496e650c2ee5831a66da02c5435c5270d6b594aec6e30511d",
2981+ "order": 0,
2982+ "path": "/daily/ubuntu/ubuntu-20130807.full.tar.xz",
2983+ "signature": "/daily/ubuntu/ubuntu-20130807.full.tar.xz.asc",
2984+ "size": 264539288
2985+ },
2986+ {
2987+ "checksum": "1a39b78b220660fe0c7fd8c30febe92009ad5306610e41f7cdf838da8290c59a",
2988+ "order": 1,
2989+ "path": "/daily/mako/mako-20130807.full.tar.xz",
2990+ "signature": "/daily/mako/mako-20130807.full.tar.xz.asc",
2991+ "size": 36875496
2992+ },
2993+ {
2994+ "checksum": "287eb50401487dc217f6a614fa03ae1bfb7fbead48f196692cf090667c98353f",
2995+ "order": 2,
2996+ "path": "/daily/mako/version-20130808.tar.xz",
2997+ "signature": "/daily/mako/version-20130808.tar.xz.asc",
2998+ "size": 192
2999+ }
3000+ ],
3001+ "type": "full",
3002+ "version": 20130808
3003+ },
3004+ {
3005+ "base": 20130807,
3006+ "description": "20130807",
3007+ "files": [
3008+ {
3009+ "checksum": "1e48af06cb25e899a536301c36f212f06c0479be69785d1569d2795099485ab5",
3010+ "order": 0,
3011+ "path": "/daily/ubuntu/ubuntu-20130807.delta-20130806.1.tar.xz",
3012+ "signature": "/daily/ubuntu/ubuntu-20130807.delta-20130806.1.tar.xz.asc",
3013+ "size": 17889552
3014+ },
3015+ {
3016+ "checksum": "10000fae3aa0a567bb10cc5efc6552bb01ee6ec96a689f0adf886ad3e202b0c7",
3017+ "order": 1,
3018+ "path": "/daily/mako/mako-20130807.delta-20130806.1.tar.xz",
3019+ "signature": "/daily/mako/mako-20130807.delta-20130806.1.tar.xz.asc",
3020+ "size": 34435132
3021+ },
3022+ {
3023+ "checksum": "287eb50401487dc217f6a614fa03ae1bfb7fbead48f196692cf090667c98353f",
3024+ "order": 2,
3025+ "path": "/daily/mako/version-20130808.tar.xz",
3026+ "signature": "/daily/mako/version-20130808.tar.xz.asc",
3027+ "size": 192
3028+ }
3029+ ],
3030+ "type": "delta",
3031+ "version": 20130808
3032+ }
3033+ ]
3034+}
3035
3036=== modified file 'tests/test_cdimage.py'
3037--- tests/test_cdimage.py 2013-07-14 20:40:06 +0000
3038+++ tests/test_cdimage.py 2013-08-09 01:41:37 +0000
3039@@ -37,7 +37,7 @@
3040 process_mock.return_value = target_build
3041 self.rsync_call.append('%s/current' % self.rsync_uri_part)
3042 # when
3043- build = cdimage.get_latest_current(self.cdimage_uri)
3044+ build = cdimage.get_build(self.cdimage_uri)
3045 # then
3046 self.assertThat(build, Equals(target_build))
3047 process_mock.assert_called_once_with(self.rsync_call)
3048@@ -49,7 +49,7 @@
3049 process_mock.return_value = target_build
3050 self.rsync_call.append('%s/pending' % self.rsync_uri_part)
3051 # when
3052- build = cdimage.get_latest_pending(self.cdimage_uri)
3053+ build = cdimage.get_build(self.cdimage_uri, pending=True)
3054 # then
3055 self.assertThat(build, Equals(target_build))
3056- process_mock.assert_called_once_with(self.rsync_call)
3057\ No newline at end of file
3058+ process_mock.assert_called_once_with(self.rsync_call)
3059
3060=== renamed file 'tests/test_environment.py' => 'tests/test_community.py'
3061--- tests/test_environment.py 2013-07-14 20:40:06 +0000
3062+++ tests/test_community.py 2013-08-09 01:41:37 +0000
3063@@ -15,134 +15,115 @@
3064
3065 """Unit tests for phabletutils.environment."""
3066
3067-from mock import patch
3068+import json
3069+import shutil
3070+import tempfile
3071+
3072 from os import path
3073-from phabletutils import environment
3074-from phabletutils import settings
3075+from phabletutils import community
3076 from testtools import TestCase
3077 from testtools.matchers import Equals
3078-from testscenarios import TestWithScenarios
3079-from testscenarios import multiply_scenarios
3080-
3081-
3082-class FakeSettings(object):
3083- download_dir = 'fake_download_dir'
3084- device_file_img = 'device-%s.img'
3085- boot_file_img = 'boot-%s.img'
3086-
3087-
3088-@patch('phabletutils.environment.get_ubuntu_stamp')
3089-class TestPlainEnvironments(TestCase):
3090- def setUp(self):
3091- super(TestPlainEnvironments, self).setUp()
3092- self.device = 'mako'
3093+from testtools.matchers import Is
3094+
3095+
3096+class TestCommunityManifestLoad(TestCase):
3097+
3098+ def setUp(self):
3099+ super(TestCommunityManifestLoad, self).setUp()
3100+ self.device_dir = tempfile.mkdtemp()
3101+
3102+ def tearDown(self):
3103+ super(TestCommunityManifestLoad, self).tearDown()
3104+ shutil.rmtree(self.device_dir)
3105+
3106+ def testSimpleDeviceManifest(self):
3107+ # given
3108+ manifest = {'device': 'http://somelocation.com/device.zip'}
3109+ with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
3110+ f.write(json.dumps(manifest))
3111+ # when
3112+ loaded_manifest = community.load_manifest(self.device_dir)
3113+ # then
3114+ self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
3115+ self.assertThat(loaded_manifest['ubuntu'], Is(None))
3116+ self.assertThat(loaded_manifest['revision'], Is(None))
3117+ self.assertThat(loaded_manifest['storage'], Equals('/sdcard/'))
3118+
3119+ def testSimpleDeviceAndUbuntuManifest(self):
3120+ # given
3121+ manifest = {'device': 'http://somelocation.com/device.zip',
3122+ 'ubuntu': 'http://somelocation.com/ubuntu.zip'}
3123+ with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
3124+ f.write(json.dumps(manifest))
3125+ # when
3126+ loaded_manifest = community.load_manifest(self.device_dir)
3127+ # then
3128+ self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
3129+ self.assertThat(loaded_manifest['ubuntu'], Equals(manifest['ubuntu']))
3130+ self.assertThat(loaded_manifest['revision'], Is(None))
3131+ self.assertThat(loaded_manifest['storage'], Equals('/sdcard/'))
3132+
3133+ def testSimpleDeviceAndUbuntuRevisionManifest(self):
3134+ # given
3135+ manifest = {'device': 'http://somelocation.com/device.zip',
3136+ 'ubuntu': 'http://somelocation.com/ubuntu.zip',
3137+ 'revision': 'unstable'}
3138+ with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
3139+ f.write(json.dumps(manifest))
3140+ # when
3141+ loaded_manifest = community.load_manifest(self.device_dir)
3142+ # then
3143+ self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
3144+ self.assertThat(loaded_manifest['ubuntu'], Equals(manifest['ubuntu']))
3145+ self.assertThat(loaded_manifest['revision'],
3146+ Equals(manifest['revision']))
3147+ self.assertThat(loaded_manifest['storage'], Equals('/sdcard/'))
3148+
3149+ def testSimpleDeviceAndUbuntuRevisionStorageManifest(self):
3150+ # given
3151+ manifest = {'device': 'http://somelocation.com/device.zip',
3152+ 'ubuntu': 'http://somelocation.com/ubuntu.zip',
3153+ 'storage': '/emmc/',
3154+ 'revision': 'unstable'}
3155+ with open(path.join(self.device_dir, 'manifest.json'), 'w') as f:
3156+ f.write(json.dumps(manifest))
3157+ # when
3158+ loaded_manifest = community.load_manifest(self.device_dir)
3159+ # then
3160+ self.assertThat(loaded_manifest['device'], Equals(manifest['device']))
3161+ self.assertThat(loaded_manifest['ubuntu'], Equals(manifest['ubuntu']))
3162+ self.assertThat(loaded_manifest['revision'],
3163+ Equals(manifest['revision']))
3164+ self.assertThat(loaded_manifest['storage'],
3165+ Equals(manifest['storage']))
3166+
3167+
3168+class TestCommunityFiles(TestCase):
3169+
3170+ def setUp(self):
3171+ super(TestCommunityFiles, self).setUp()
3172+ self.download_dir = '/downloads'
3173 self.series = 'saucy'
3174- self.settings = FakeSettings()
3175- self.uri = 'http://download.com/artifacts'
3176- self.build = '10'
3177-
3178- def testNonCdimage(self, get_ubuntu_stamp):
3179- # given
3180- get_ubuntu_stamp.return_value = self.build
3181- download_dir = path.join(self.settings.download_dir, self.build)
3182- full_download_dir = environment.get_download_dir_full_path(
3183- download_dir)
3184- system_img_path = path.join(full_download_dir,
3185- self.settings.device_file_img % self.device)
3186- boot_img_path = path.join(full_download_dir,
3187- self.settings.boot_file_img % self.device)
3188- # when
3189- env = environment.Environment(self.uri, self.device, self.series,
3190- None, None, False, None, None,
3191- self.settings)
3192- # then
3193- self.assertFalse(env.project)
3194- self.assertThat(env.download_dir, Equals(full_download_dir))
3195- self.assertThat(env.download_uri, Equals(self.uri))
3196- self.assertThat(env.system_img_path, Equals(system_img_path))
3197- self.assertThat(env.boot_img_path, Equals(boot_img_path))
3198- self.assertFalse(env.recovery_img_path)
3199-
3200-
3201-def get_file_paths(files, download_dir, device, series):
3202- if not series:
3203- series = settings.default_series
3204- replacements = (series, device)
3205- files = {
3206- 'system_img': path.join(download_dir,
3207- files['system_img'] % replacements),
3208- 'boot_img': path.join(download_dir,
3209- files['boot_img'] % replacements),
3210- 'recovery_img': path.join(download_dir,
3211- files['recovery_img'] % replacements),
3212- 'device_zip': path.join(download_dir,
3213- files['device_zip'] % replacements),
3214- 'ubuntu_zip': path.join(download_dir, files['ubuntu_zip'] % series),
3215- }
3216- return files
3217-
3218-
3219-@patch('phabletutils.cdimage.get_sha256_content')
3220-@patch('phabletutils.cdimage.get_latest_current')
3221-class TestCdimageEnvironments(TestWithScenarios, TestCase):
3222-
3223- scenarios = multiply_scenarios(
3224- [('legacy touch', dict(legacy=True)),
3225- ('touch', dict(legacy=False))],
3226- [('use default series', dict(series=None)),
3227- ('set series', dict(series='raring'))],
3228- [('base path', dict(base_path='/prev_download')),
3229- ('no base path', dict(base_path=None))])
3230-
3231- def setUp(self):
3232- super(TestCdimageEnvironments, self).setUp()
3233- if self.legacy:
3234- self.project = 'ubuntu-touch-preview'
3235- else:
3236- self.project = 'ubuntu-touch'
3237- self.device = 'manta'
3238- self.build = '20130606'
3239- self.download_dir = path.join(settings.download_dir,
3240- self.project, self.build)
3241- if not self.base_path:
3242- self.full_download_dir = environment.get_download_dir_full_path(
3243- self.download_dir)
3244- self.download_uri = path.join(settings.cdimage_uri_base,
3245- self.project, 'daily-preinstalled', self.build)
3246- else:
3247- self.full_download_dir = self.base_path
3248- self.download_uri = None
3249- self.files = get_file_paths(settings.files[self.project],
3250- self.full_download_dir,
3251- self.device, self.series)
3252-
3253- def testEnvironmentSetup(self, cdimage_get_latest, cdimage_sha256):
3254- # given
3255- cdimage_get_latest.return_value = self.build
3256- cdimage_sha256.return_value = 'sha'
3257- # when
3258- env = environment.Environment(None, self.device, self.series,
3259- None, None, False, self.legacy,
3260- self.base_path, settings)
3261- # then
3262- self.assertTrue(env.project)
3263- self.assertThat(env.download_dir, Equals(self.full_download_dir))
3264- self.assertThat(env.download_uri, Equals(self.download_uri))
3265- self.assertThat(env.system_img_path, Equals(self.files['system_img']))
3266- self.assertThat(env.boot_img_path, Equals(self.files['boot_img']))
3267- self.assertThat(env.device_zip_path, Equals(self.files['device_zip']))
3268- self.assertThat(env.ubuntu_zip_path, Equals(self.files['ubuntu_zip']))
3269- self.assertThat(env.recovery_img_path,
3270- Equals(self.files['recovery_img']))
3271- # Check recovery file list
3272- self.assertTrue(self.files['device_zip'] in env.recovery_files)
3273- self.assertTrue(self.files['ubuntu_zip'] in env.recovery_files)
3274- self.assertFalse(self.files['recovery_img'] in env.recovery_files)
3275- self.assertFalse(self.files['system_img'] in env.recovery_files)
3276- self.assertFalse(self.files['boot_img'] in env.recovery_files)
3277- # Check bootstrap file list
3278- self.assertFalse(self.files['device_zip'] in env.bootstrap_files)
3279- self.assertTrue(self.files['ubuntu_zip'] in env.bootstrap_files)
3280- self.assertTrue(self.files['recovery_img'] in env.bootstrap_files)
3281- self.assertTrue(self.files['system_img'] in env.bootstrap_files)
3282- self.assertTrue(self.files['boot_img'] in env.bootstrap_files)
3283\ No newline at end of file
3284+
3285+ def tearDown(self):
3286+ super(TestCommunityFiles, self).tearDown()
3287+
3288+ def testSimpleDeviceAndUbuntu(self):
3289+ # given
3290+ manifest = {'device': 'http://somelocation.com/device.zip',
3291+ 'ubuntu': 'http://somelocation.com/ubuntu.zip'}
3292+ # when
3293+ files = community.get_files(manifest, self.download_dir, self.series)
3294+ #then
3295+ self.assertThat(files['device'].uri, Equals(manifest['device']))
3296+ self.assertThat(files['device'].hash, Is(None))
3297+ self.assertThat(files['device'].verified, Is(False))
3298+ self.assertThat(files['device'].path,
3299+ Equals(path.join(self.download_dir,
3300+ path.basename(manifest['device']))))
3301+ self.assertThat(files['ubuntu'].uri, Equals(manifest['ubuntu']))
3302+ self.assertThat(files['ubuntu'].hash, Is(None))
3303+ self.assertThat(files['ubuntu'].verified, Is(False))
3304+ self.assertThat(files['ubuntu'].path,
3305+ Equals(path.join(self.download_dir,
3306+ path.basename(manifest['ubuntu']))))
3307
3308=== added file 'tests/test_ubuntuimage.py'
3309--- tests/test_ubuntuimage.py 1970-01-01 00:00:00 +0000
3310+++ tests/test_ubuntuimage.py 2013-08-09 01:41:37 +0000
3311@@ -0,0 +1,54 @@
3312+# Copyright (C) 2013 Canonical Ltd.
3313+# Author: Sergio Schvezov <sergio.schvezov@canonical.com>
3314+
3315+# This program is free software: you can redistribute it and/or modify
3316+# it under the terms of the GNU General Public License as published by
3317+# the Free Software Foundation; version 3 of the License.
3318+#
3319+# This program is distributed in the hope that it will be useful,
3320+# but WITHOUT ANY WARRANTY; without even the implied warranty of
3321+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
3322+# GNU General Public License for more details.
3323+#
3324+# You should have received a copy of the GNU General Public License
3325+# along with this program. If not, see <http://www.gnu.org/licenses/>.
3326+
3327+"""Unit tests for phabletutils.environment."""
3328+
3329+from mock import patch
3330+from phabletutils import ubuntuimage
3331+from testtools import TestCase
3332+from testtools.matchers import Equals
3333+from testtools.matchers import HasLength
3334+
3335+
3336+@patch('phabletutils.downloads.get_content')
3337+class TestUbuntuImage(TestCase):
3338+
3339+ def setUp(self):
3340+ super(TestUbuntuImage, self).setUp()
3341+ self.device = 'mako'
3342+ with open('tests/index.json', 'r') as f:
3343+ self.json_content = f.read()
3344+
3345+ def testGetLatestIndex(self, get_content_mock):
3346+ # given
3347+ get_content_mock.return_value = self.json_content
3348+ # when
3349+ json_dict = ubuntuimage.get_json_from_index(self.device, 0)
3350+ # then
3351+ self.assertThat(json_dict['version'], Equals(20130808))
3352+ self.assertThat(json_dict['description'], Equals('20130807'))
3353+ self.assertThat(json_dict['type'], Equals('full'))
3354+ self.assertThat(json_dict['files'], HasLength(3))
3355+
3356+ def testGetPreviousIndex(self, get_content_mock):
3357+ # given
3358+ get_content_mock.return_value = self.json_content
3359+ # when
3360+ json_dict = ubuntuimage.get_json_from_index(self.device, -1)
3361+ # then
3362+ self.assertThat(json_dict['version'], Equals(20130807))
3363+ self.assertThat(json_dict['description'], Equals('20130806.1'))
3364+ self.assertThat(json_dict['type'], Equals('full'))
3365+ self.assertThat(json_dict['files'], HasLength(3))

Subscribers

People subscribed via source and target branches