Merge lp:~frankban/lpsetup/complete-tests into lp:lpsetup

Proposed by Francesco Banconi on 2012-07-20
Status: Merged
Approved by: Francesco Banconi on 2012-07-20
Approved revision: 74
Merged at revision: 58
Proposed branch: lp:~frankban/lpsetup/complete-tests
Merge into: lp:lpsetup
Diff against target: 1050 lines (+309/-205)
15 files modified
lpsetup/handlers.py (+51/-21)
lpsetup/subcommands/finish_inithost.py (+11/-28)
lpsetup/subcommands/inithost.py (+48/-26)
lpsetup/subcommands/initlxc.py (+10/-11)
lpsetup/subcommands/initrepo.py (+10/-1)
lpsetup/subcommands/install_lxc.py (+24/-25)
lpsetup/subcommands/update.py (+24/-37)
lpsetup/tests/subcommands/test_finish_inithost.py (+4/-9)
lpsetup/tests/subcommands/test_inithost.py (+19/-18)
lpsetup/tests/subcommands/test_initlxc.py (+1/-0)
lpsetup/tests/subcommands/test_initrepo.py (+2/-11)
lpsetup/tests/subcommands/test_install_lxc.py (+5/-5)
lpsetup/tests/subcommands/test_update.py (+8/-11)
lpsetup/tests/test_handlers.py (+81/-2)
lpsetup/tests/utils.py (+11/-0)
To merge this branch: bzr merge lp:~frankban/lpsetup/complete-tests
Reviewer Review Type Date Requested Status
Brad Crittenden (community) code 2012-07-20 Approve on 2012-07-20
Review via email: mp+115985@code.launchpad.net

Commit Message

Commands updated to reflect the new proposed UI and to support the parallel tests story (buildbot, http, no-checkout).

Description of the Change

Sorry, the diff is huge, I had to change several things to support the "buildbot-http-no-checkout" story, but at least manual tests pass, and *lpsetup install-lxc* exits successfully when run like the following::

    lp-setup install-lxc -B `bzr info | grep parent | cut -c 18-`
    -u buildbot -E <email address hidden> -f 'Launchpad PQM' -r
    /var/lib/buildbot/slaves/slave --no-checkout -S launchpad_lxc_id_rsa
    --use-http

== Changes ==

Changed UI for *finish_inithost* and *update* commands, as described in my email this morning: they now accept a *target_dir* defaulting to current directory.

handle_lpuser_from_lplogin: the handler no longer raises a ValidationError if --use-http is provided.

handle_target_from_repository: added an handler to calculate the *target_dir*. A target_dir must be passed to both update and finish-inithost commands.

handle_target_dir: just makes namespace.target_dir an absolute path.

Renamed *code_dir* and *working_dir* to *target_dir*.

s/ensure_ssh_keys/setup_ssh: it seemed to me more expressive. Also updated names in the relevant test case.

Added a *setup_home* step to init-host: from the docstring::

     This is a separate step for several reasons::
        - It is directly called by *initlxc* as one of its first steps.
          Note that *initlxc* needs the user's `.ssh` dir to be set up
          to be able to connect to the container.
        - *initlxc* should be able to skip this step when re-executing
          *inithost* from inside the container, to avoid setting up the
          user home two times. Note: the user home is bind mounted by lxc.

Consequentially `changed initlxc.inithost_in_lxc` to skip the *setup_home* step.

inithost: moved the code calling *setup_ssh* and *bzr whoami/lp-login* from *initialize_base* to *initialize*, so that the ssh directory and bzr login are set up only once by initlxc.

initlxc: moved initialize() up at the beginning of the file so that it's easier to follow the code flow.

initrepo: setup_bzr_locations is now only called when the user Launchpad login can be retrieved. The check is done defining a call_setup_bzr_locations subcommand method.

install_lxc: steps updated to reflect the new UI. Added support for --no-checkout in *init_repo_in_lxc*. Updated *cmd_in_lxc* to ssh as the given user rather than ssh as root and then run sudo.

Moved lpuser and skip_if_no_lpuser definition from `tests.subcommands.test_initrepo` to `tests.utils` because they are now used by other tests too (see below).

Added a testcase for `handlers.handle_lpuser_from_lplogin`.

To post a comment you must log in.
Brad Crittenden (bac) wrote :

Hi Francesco,

* For handlers it would be nice to note any dependencies, e.g. handle_target_dir could note that it must be listed after handle_target_from_repository, otherwise namespace.target_dir

* At line 175 multi-line docstrings should close on a separate line.

* Did you intend to leave the output CMD= at line 393?

* Typo at line 900 /happen/happens/

Otherwise this branch looks good and is a nice change to make the cli nice for 'update'. Thanks.

review: Approve (code)
lp:~frankban/lpsetup/complete-tests updated on 2012-07-20
71. By Francesco Banconi on 2012-07-20

Multi line docstring fix.

72. By Francesco Banconi on 2012-07-20

Remove print statement.

73. By Francesco Banconi on 2012-07-20

Fixed typo.

74. By Francesco Banconi on 2012-07-20

Better docstring for handlers. Removed unused function.

Francesco Banconi (frankban) wrote :

Thanks Brad.
Branch updated following your suggestions.
Also removed the "print CMD..." line.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lpsetup/handlers.py'
2--- lpsetup/handlers.py 2012-07-19 17:48:40 +0000
3+++ lpsetup/handlers.py 2012-07-20 15:13:48 +0000
4@@ -6,11 +6,14 @@
5
6 __metaclass__ = type
7 __all__ = [
8- 'handle_code_dir',
9+ 'handle_branch_and_checkout',
10 'handle_directories',
11 'handle_lpuser_as_username',
12 'handle_lpuser_from_lplogin',
13+ 'handle_source',
14 'handle_ssh_keys',
15+ 'handle_target_dir',
16+ 'handle_target_from_repository',
17 'handle_testing',
18 'handle_user',
19 'handle_userdata',
20@@ -68,7 +71,10 @@
21 def handle_lpuser_as_username(namespace):
22 """Handle lpuser argument.
23
24- If lpuser is not provided by namespace, the user name is used::
25+ The namespace must contain *user* and *lpuser*.
26+ It should be invoked after handle_user.
27+
28+ If lpuser is None, the user name is used::
29
30 >>> import getpass
31 >>> username = getpass.getuser()
32@@ -85,12 +91,18 @@
33
34 def handle_lpuser_from_lplogin(namespace):
35 """Handle lpuser argument.
36+
37+ The validation fails if *lpuser* is not provided, is not retrievable
38+ and *use-http* is False.
39 """
40 if getattr(namespace, 'lpuser', None) is None:
41 try:
42 namespace.lpuser = run('bzr', 'launchpad-login').strip()
43 except subprocess.CalledProcessError:
44- raise ValidationError("No bzr launchpad-login set.")
45+ if getattr(namespace, 'use_http', False):
46+ namespace.lpuser = None
47+ else:
48+ raise ValidationError("No bzr launchpad-login set.")
49
50
51 def handle_userdata(namespace, whois=bzr_whois):
52@@ -153,6 +165,9 @@
53 def handle_ssh_keys(namespace):
54 r"""Handle private and public ssh keys.
55
56+ The namespace must contain *home_dir* and *ssh_key_name*.
57+ It should be invoked after handle_user.
58+
59 Keys contained in the namespace are escaped::
60
61 >>> import argparse
62@@ -224,6 +239,9 @@
63 def handle_directories(namespace):
64 """Handle repository and dependencies directories.
65
66+ The namespace must contain *home_dir*.
67+ It should be invoked after handle_user.
68+
69 - The ~ construction is automatically expanded.
70 - The validation fails for directories not residing inside the home.
71 - The validation fails if the directory contains spaces.
72@@ -266,7 +284,10 @@
73
74
75 def handle_source(namespace):
76- """Handle the --source and --use-http options."""
77+ """Handle the --source and --use-http options.
78+
79+ The namespace must contain *source* or *use_http*.
80+ """
81 # If a source has been provided, then there is nothing else to do.
82 if namespace.source is not None:
83 return
84@@ -276,14 +297,32 @@
85 namespace.source = LP_SSH_REPO
86
87
88-def normalize_path(path):
89- """Return the absolute, expanded path.
90-
91- If none given, expand the current working directory.
92- """
93- if not path:
94- path = '.'
95- return os.path.abspath(os.path.expanduser(path))
96+def handle_target_from_repository(namespace):
97+ """Handle *repository*, *branch_name*, *checkout_name* and *no_checkout*.
98+
99+ The namespace must contain *repository*, *branch_name*,
100+ *checkout_name* and *no_checkout*.
101+ It should be invoked after handle_directories, and when *target_dir* is
102+ not explicitly passed to a command.
103+
104+ These names must be present in the namespace.
105+ Produces *target_dir*: the real working dir where the actual code lives.
106+ """
107+ if namespace.no_checkout:
108+ target_name = namespace.branch_name
109+ else:
110+ target_name = namespace.checkout_name
111+ namespace.target_dir = os.path.join(namespace.repository, target_name)
112+
113+
114+def handle_target_dir(namespace):
115+ """Handle path to the working directory.
116+
117+ The namespace must contain *target_dir*.
118+ It should be invoked when *target_dir* is explicitly passed to a command.
119+ """
120+ path = namespace.target_dir
121+ namespace.target_dir = os.path.abspath(os.path.expanduser(path))
122
123
124 def handle_branch_and_checkout(namespace):
125@@ -315,12 +354,3 @@
126 raise ValidationError(
127 'branch and checkout: can not use the same name ({0}).'.format(
128 checkout_name))
129-
130-
131-def handle_code_dir(namespace):
132- """Handle the computed value for `code_dir`.
133-
134- It must be invoked after handle_directories.
135- """
136- namespace.code_dir = os.path.join(
137- namespace.repository, namespace.checkout_name)
138
139=== modified file 'lpsetup/subcommands/finish_inithost.py'
140--- lpsetup/subcommands/finish_inithost.py 2012-07-19 17:48:40 +0000
141+++ lpsetup/subcommands/finish_inithost.py 2012-07-20 15:13:48 +0000
142@@ -29,28 +29,22 @@
143
144 from lpsetup import argparser
145 from lpsetup.handlers import (
146- handle_code_dir,
147- handle_directories,
148 handle_user,
149- )
150-from lpsetup.settings import (
151- LP_CHECKOUT_NAME,
152- LP_REPOSITORY_DIR,
153- )
154-
155+ handle_target_dir,
156+ )
157 from lpsetup.utils import call
158
159
160-def setup_launchpad(user, code_dir):
161+def setup_launchpad(user, target_dir):
162 """Set up the Launchpad environment."""
163
164 # Launchpad database setup.
165 # nested is required for use by python 2.6.
166- with nested(su(user), cd(code_dir)):
167+ with nested(su(user), cd(target_dir)):
168 call('utilities/launchpad-database-setup', user)
169
170 # Make and install launchpad.
171- with cd(code_dir):
172+ with cd(target_dir):
173 # Using real su because mailman make script uses uid.
174 call(*get_su_command(user, ['make', 'schema']))
175 call('make', 'install')
176@@ -74,30 +68,19 @@
177 needs_root = True
178
179 steps = (
180- (setup_launchpad, 'user', 'code_dir'),
181+ (setup_launchpad, 'user', 'target_dir'),
182 )
183
184- handlers = (
185- handle_user,
186- handle_directories,
187- handle_code_dir,
188- )
189+ handlers = (handle_user, handle_target_dir)
190
191 def add_arguments(self, parser):
192 super(SubCommand, self).add_arguments(parser)
193 parser.add_argument(
194+ 'target_dir', nargs='?', default=os.getcwd(),
195+ help='The directory of the Launchpad code checkout. '
196+ '[DEFAULT=current directory]')
197+ parser.add_argument(
198 '-u', '--user',
199 help='The name of the system user. '
200 'The current user is used if this script is not run as '
201 'root and this argument is omitted.')
202- parser.add_argument(
203- '--checkout-name', default=LP_CHECKOUT_NAME,
204- help='Create a checkout with the given name. '
205- 'Ignored if --no-checkout is specified. '
206- 'Defaults to {0}.'.format(LP_CHECKOUT_NAME))
207- parser.add_argument(
208- '-r', '--repository', default=LP_REPOSITORY_DIR,
209- help='The directory of the Launchpad repository to be created. '
210- 'The directory must reside under the home directory of the '
211- 'given user (see -u argument). '
212- '[DEFAULT={0}]'.format(LP_REPOSITORY_DIR))
213
214=== modified file 'lpsetup/subcommands/inithost.py'
215--- lpsetup/subcommands/inithost.py 2012-07-13 15:16:12 +0000
216+++ lpsetup/subcommands/inithost.py 2012-07-20 15:13:48 +0000
217@@ -99,9 +99,11 @@
218 os.chmod(filename, 0644)
219
220
221-def ensure_ssh_keys(ssh_dir, private_key, public_key, valid_ssh_keys,
222- ssh_key_path):
223- """Ensure that SSH is configured correctly for the user."""
224+def setup_ssh(ssh_dir, private_key, public_key, valid_ssh_keys, ssh_key_path):
225+ """Set up the user's `.ssh` directory.
226+
227+ Also ensure that SSH is configured correctly for the user.
228+ """
229 # Set up the user's ssh directory. The ssh key must be associated
230 # with the lpuser's Launchpad account.
231 mkdirs(ssh_dir)
232@@ -138,29 +140,12 @@
233 if not user_exists(user):
234 call('useradd', '-m', '-s', '/bin/bash', '-U', user)
235
236- # Add user to the sudo group.
237- subprocess.call(['adduser', user, 'sudo'])
238- # Add the user to his own group.
239- pwd_database = pwd.getpwnam(user)
240- subprocess.call(['addgroup', '--gid', str(pwd_database.pw_gid), user])
241-
242-
243-def initialize(
244- user, full_name, email, lpuser, private_key, public_key, valid_ssh_keys,
245- ssh_key_path):
246+
247+def initialize(user):
248 """Initialize host machine."""
249 make_version_dir()
250 initialize_base(user)
251- with su(user) as env:
252- ssh_dir = os.path.join(env.home, '.ssh')
253- ensure_ssh_keys(
254- ssh_dir, private_key, public_key, valid_ssh_keys, ssh_key_path)
255-
256- # Set up bzr and Launchpad authentication.
257- call('bzr', 'whoami', formataddr([full_name, email]))
258- if valid_ssh_keys:
259- subprocess.call(['bzr', 'lp-login', lpuser])
260-
261+ with su(user):
262 # Create Apache document roots, to avoid warnings.
263 mkdirs(*LP_APACHE_ROOTS)
264
265@@ -168,6 +153,15 @@
266 for module in LP_APACHE_MODULES.split():
267 call('a2enmod', module)
268
269+ # The user must be added to sudoers if inithost is run in a container
270+ # and we are running the developer story.
271+ # XXX 2012-07-18 frankban: add the developer/testing check.
272+ if running_in_container():
273+ subprocess.call(['adduser', user, 'sudo'])
274+ # Add the user to his own group.
275+ pwd_database = pwd.getpwnam(user)
276+ subprocess.call(['addgroup', '--gid', str(pwd_database.pw_gid), user])
277+
278 # Set up container hosts file.
279 lines = [get_file_header()]
280 lines.extend(['{0}\t{1}\n'.format(ip, names)
281@@ -177,6 +171,32 @@
282 file_append(HOSTS_FILE, line)
283
284
285+def setup_home(
286+ user, full_name, email, lpuser, private_key, public_key, valid_ssh_keys,
287+ ssh_key_path):
288+ """Initialize the user home directory.
289+
290+ This is a separate step for several reasons::
291+
292+ - It is directly called by *initlxc* as one of its first steps.
293+ Note that *initlxc* needs the user's `.ssh` dir to be set up
294+ to be able to connect to the container.
295+ - *initlxc* should be able to skip this step when re-executing
296+ *inithost* from inside the container, to avoid setting up the
297+ user home two times. Note: the user home is bind mounted by lxc.
298+ """
299+ with su(user) as env:
300+ # Set up the `.ssh` directory.
301+ setup_ssh(
302+ os.path.join(env.home, '.ssh'), private_key, public_key,
303+ valid_ssh_keys, ssh_key_path)
304+
305+ # Set up bzr and Launchpad authentication.
306+ call('bzr', 'whoami', formataddr([full_name, email]))
307+ if valid_ssh_keys:
308+ subprocess.call(['bzr', 'lp-login', lpuser])
309+
310+
311 def initialize_lxc():
312 """Initialize LXC container.
313
314@@ -223,10 +243,11 @@
315
316 class SubCommand(argparser.StepsBasedSubCommand):
317 """Prepare a machine to run Launchpad. May be an LXC container or not."""
318- initialize_step = (initialize,
319+ initialize_step = (initialize, 'user')
320+
321+ setup_home_step = (setup_home,
322 'user', 'full_name', 'email', 'lpuser',
323- 'private_key', 'public_key', 'valid_ssh_keys', 'ssh_key_path',
324- )
325+ 'private_key', 'public_key', 'valid_ssh_keys', 'ssh_key_path')
326
327 initialize_lxc_step = (initialize_lxc, )
328
329@@ -234,6 +255,7 @@
330
331 steps = (
332 initialize_step,
333+ setup_home_step,
334 initialize_lxc_step,
335 setup_apt_step,
336 )
337
338=== modified file 'lpsetup/subcommands/initlxc.py'
339--- lpsetup/subcommands/initlxc.py 2012-07-17 20:17:48 +0000
340+++ lpsetup/subcommands/initlxc.py 2012-07-20 15:13:48 +0000
341@@ -57,6 +57,14 @@
342 )
343
344
345+def initialize(user):
346+ """Initialize the LXC host."""
347+ inithost.initialize_base(user)
348+ # haveged is used to fill /dev/random, avoiding
349+ # entropy exhaustion during automated parallel tests.
350+ apt_get_install('haveged', caller=call)
351+
352+
353 def create_lxc(lxc_name, lxc_arch, lxc_os, user, install_subunit=False):
354 """Create the LXC named `lxc_name` sharing `user` home directory.
355
356@@ -125,14 +133,6 @@
357 retry_ssh(lxc_name, 'true', key=ssh_key_path)
358
359
360-def initialize(user):
361- """Initialize the LXC host."""
362- inithost.initialize_base(user)
363- # haveged is used to fill /dev/random, avoiding
364- # entropy exhaustion during automated parallel tests.
365- apt_get_install('haveged', caller=call)
366-
367-
368 def install_lpsetup_in_lxc(lxc_name, ssh_key_path, lxc_os,
369 user, home_dir, lpsetup_branch=None):
370 """Initialize LXC container."""
371@@ -194,9 +194,7 @@
372 """Prepare the Launchpad environment inside an LXC."""
373 # Use ssh to call this script from inside the container.
374 args = ['init-host', '-u', user, '-E', email, '-f', full_name,
375- '-l', lpuser, '-S', ssh_key_name,
376- ]
377-
378+ '-l', lpuser, '-S', ssh_key_name, '--skip-steps', 'setup_home']
379 cmd = this_command(home_dir, args)
380 ssh(lxc_name, cmd, key=ssh_key_path)
381
382@@ -230,6 +228,7 @@
383
384 steps = (
385 (initialize, 'user'),
386+ inithost.SubCommand.setup_home_step,
387 create_lxc_step,
388 start_lxc_step,
389 wait_for_lxc_step,
390
391=== modified file 'lpsetup/subcommands/initrepo.py'
392--- lpsetup/subcommands/initrepo.py 2012-07-19 14:31:26 +0000
393+++ lpsetup/subcommands/initrepo.py 2012-07-20 15:13:48 +0000
394@@ -86,7 +86,11 @@
395
396 def setup_bzr_locations(
397 lpuser, repository, branch_name, template=LP_BZR_LOCATIONS):
398- """Set up bazaar locations."""
399+ """Set up bazaar locations.
400+
401+ Note that this step is guarded by the *call_setup_bzr_locations* method
402+ so that it is only called when the user Launchpad login can be retrieved.
403+ """
404 context = {
405 'branch_dir': os.path.join(repository, branch_name),
406 'repository': repository,
407@@ -128,6 +132,11 @@
408 handlers.handle_source,
409 )
410
411+ def call_setup_bzr_locations(self, namespace, step, args):
412+ """Caller that only sets up bzr locations if lpuser exists."""
413+ if namespace.lpuser is not None:
414+ return step(*args)
415+
416 @staticmethod
417 def add_common_arguments(parser):
418 parser.add_argument(
419
420=== modified file 'lpsetup/subcommands/install_lxc.py'
421--- lpsetup/subcommands/install_lxc.py 2012-07-19 17:48:40 +0000
422+++ lpsetup/subcommands/install_lxc.py 2012-07-20 15:13:48 +0000
423@@ -13,9 +13,10 @@
424 import os
425
426 from lpsetup.handlers import (
427- handle_code_dir,
428+ handle_branch_and_checkout,
429 handle_directories,
430 handle_source,
431+ handle_target_from_repository,
432 )
433 from lpsetup.settings import (
434 LXC_IP_COMMAND,
435@@ -70,13 +71,12 @@
436
437 def cmd_in_lxc(lxc_name, ssh_key_path, home_dir, args, as_user=None):
438 cmd = this_command(home_dir, args)
439- if as_user is not None:
440- cmd = "su {} -c '{}'".format(as_user, cmd)
441- ssh(lxc_name, cmd, key=ssh_key_path)
442-
443-
444-def init_repo_in_lxc(lxc_name, ssh_key_path, home_dir, user, source,
445- use_http, branch_name, checkout_name, repository):
446+ ssh(lxc_name, cmd, key=ssh_key_path, user=as_user)
447+
448+
449+def init_repo_in_lxc(
450+ lxc_name, ssh_key_path, home_dir, user, source, use_http,
451+ branch_name, checkout_name, repository, no_checkout):
452 args = [
453 'init-repo', '--source', source,
454 '--branch-name', branch_name, '--checkout-name', checkout_name,
455@@ -84,26 +84,23 @@
456 ]
457 if use_http:
458 args.append('--use-http')
459+ if no_checkout:
460+ args.append('--no-checkout')
461 cmd_in_lxc(lxc_name, ssh_key_path, home_dir, args, as_user=user)
462
463
464-def update_in_lxc(lxc_name, ssh_key_path, home_dir, user, external_path,
465- use_http, checkout_name, repository):
466- args = [
467- 'update', '--external-path', external_path,
468- '-r', repository, '--checkout-name', checkout_name,
469- ]
470+def update_in_lxc(
471+ lxc_name, ssh_key_path, home_dir, user, external_path,
472+ target_dir, use_http):
473+ args = ['update', target_dir, '--external-path', external_path]
474 if use_http:
475 args.append('--use-http')
476 cmd_in_lxc(lxc_name, ssh_key_path, home_dir, args, as_user=user)
477
478
479-def finish_inithost_in_lxc(lxc_name, ssh_key_path, home_dir, user,
480- repository, checkout_name):
481- args = [
482- 'finish-init-host', '--user', user, '--repository', repository,
483- '--checkout-name', checkout_name,
484- ]
485+def finish_inithost_in_lxc(
486+ lxc_name, ssh_key_path, home_dir, user, target_dir):
487+ args = ['finish-init-host', target_dir, '--user', user]
488 cmd_in_lxc(lxc_name, ssh_key_path, home_dir, args)
489
490
491@@ -116,12 +113,13 @@
492 # Run on host:
493 (create_scripts, 'lxc_name', 'ssh_key_path', 'user'),
494 # Run inside the container:
495- (init_repo_in_lxc, 'lxc_name', 'ssh_key_path', 'home_dir', 'user',
496- 'source', 'use_http', 'branch_name', 'checkout_name', 'repository'),
497+ (init_repo_in_lxc,
498+ 'lxc_name', 'ssh_key_path', 'home_dir', 'user', 'source', 'use_http',
499+ 'branch_name', 'checkout_name', 'repository', 'no_checkout'),
500 (update_in_lxc, 'lxc_name', 'ssh_key_path', 'home_dir', 'user',
501- 'external_path', 'use_http', 'checkout_name', 'repository'),
502+ 'external_path', 'target_dir', 'use_http'),
503 (finish_inithost_in_lxc, 'lxc_name', 'ssh_key_path', 'home_dir',
504- 'user', 'repository', 'checkout_name'),
505+ 'user', 'target_dir'),
506 )
507
508 help = __doc__
509@@ -130,8 +128,9 @@
510 handlers = super(SubCommand, self).get_handlers(namespace)
511 return handlers + (
512 handle_directories,
513- handle_code_dir,
514+ handle_branch_and_checkout,
515 handle_source,
516+ handle_target_from_repository,
517 )
518
519 def call_create_scripts(self, namespace, step, args):
520
521=== modified file 'lpsetup/subcommands/update.py'
522--- lpsetup/subcommands/update.py 2012-07-19 17:48:40 +0000
523+++ lpsetup/subcommands/update.py 2012-07-20 15:13:48 +0000
524@@ -18,50 +18,46 @@
525
526 from lpsetup import argparser
527 from lpsetup import handlers
528-from lpsetup.settings import (
529- LP_CHECKOUT_NAME,
530- LP_REPOSITORY_DIR,
531- LP_SOURCE_DEPS,
532- )
533-
534-
535-def initialize_directories(code_dir, external_path):
536+from lpsetup.settings import LP_SOURCE_DEPS
537+
538+
539+def initialize_directories(target_dir, external_path):
540 """Initialize the eggs, yui, and sourcecode directories.
541
542 Create them if necessary.
543 """
544 for dir_ in ['eggs', 'yui', 'sourcecode']:
545- mkdirs(os.path.join(code_dir, external_path, dir_))
546-
547-
548-def update_dependencies(code_dir, external_path, use_http):
549+ mkdirs(os.path.join(target_dir, external_path, dir_))
550+
551+
552+def update_dependencies(target_dir, external_path, use_http):
553 """Update the external dependencies."""
554 use_http_param = '--use-http' if use_http else None
555- cmd = os.path.join(code_dir, 'utilities', 'update-sourcecode')
556+ cmd = os.path.join(target_dir, 'utilities', 'update-sourcecode')
557 abs_external_path = os.path.abspath(
558- os.path.join(code_dir, external_path))
559+ os.path.join(target_dir, external_path))
560 source_path = os.path.join(abs_external_path, 'sourcecode')
561 run(cmd, use_http_param, source_path)
562
563 # Update the download cache.
564- download_cache = os.path.join(code_dir, 'download-cache')
565+ download_cache = os.path.join(target_dir, 'download-cache')
566 if os.path.exists(download_cache):
567 run('bzr', 'up', download_cache)
568 else:
569 run('bzr', 'co', '-v', '--lightweight', LP_SOURCE_DEPS, download_cache)
570
571 # Link to the external sourcecode.
572- if abs_external_path != code_dir:
573+ if abs_external_path != target_dir:
574 cmd = os.path.join(
575- code_dir, 'utilities', 'link-external-sourcecode')
576+ target_dir, 'utilities', 'link-external-sourcecode')
577 run(cmd,
578- '--target', code_dir,
579+ '--target', target_dir,
580 '--parent', external_path)
581
582
583-def update_tree(code_dir):
584- """Update the tree at code_dir with the latest LP code."""
585- with cd(code_dir):
586+def update_tree(target_dir):
587+ """Update the tree at target_dir with the latest LP code."""
588+ with cd(target_dir):
589 run('bzr', 'pull')
590
591
592@@ -72,16 +68,15 @@
593 """
594
595 steps = (
596- (initialize_directories, 'code_dir', 'external_path'),
597- (update_dependencies, 'code_dir', 'external_path', 'use_http'),
598- (update_tree, 'code_dir'),
599+ (initialize_directories, 'target_dir', 'external_path'),
600+ (update_dependencies, 'target_dir', 'external_path', 'use_http'),
601+ (update_tree, 'target_dir'),
602 )
603 help = __doc__
604 handlers = (
605 # Normalize paths and default to cwd if none exists.
606 handlers.handle_user,
607- handlers.handle_directories,
608- handlers.handle_code_dir,
609+ handlers.handle_target_dir,
610 )
611
612 @staticmethod
613@@ -94,19 +89,11 @@
614
615 def add_arguments(self, parser):
616 super(SubCommand, self).add_arguments(parser)
617+ parser.add_argument(
618+ 'target_dir', nargs='?', default=os.getcwd(),
619+ help='Path to branch to update. [DEFAULT=current directory]')
620 self.add_common_arguments(parser)
621 parser.add_argument(
622 '--use-http', default=False, action='store_true',
623 help='Force bzr to use http to get the sourcecode '
624 'branches rather than using bzr+ssh.')
625- parser.add_argument(
626- '--checkout-name', default=LP_CHECKOUT_NAME,
627- help='Create a checkout with the given name. '
628- 'Ignored if --no-checkout is specified. '
629- 'Defaults to {0}.'.format(LP_CHECKOUT_NAME))
630- parser.add_argument(
631- '-r', '--repository', default=LP_REPOSITORY_DIR,
632- help='The directory of the Launchpad repository to be created. '
633- 'The directory must reside under the home directory of the '
634- 'given user (see -u argument). '
635- '[DEFAULT={0}]'.format(LP_REPOSITORY_DIR))
636
637=== modified file 'lpsetup/tests/subcommands/test_finish_inithost.py'
638--- lpsetup/tests/subcommands/test_finish_inithost.py 2012-07-19 17:48:40 +0000
639+++ lpsetup/tests/subcommands/test_finish_inithost.py 2012-07-20 15:13:48 +0000
640@@ -15,25 +15,20 @@
641
642
643 setup_launchpad_step = (
644- finish_inithost.setup_launchpad, ['user', 'code_dir'])
645+ finish_inithost.setup_launchpad, ['user', 'target_dir'])
646
647
648 def get_arguments():
649- repo = '~/' + get_random_string()
650- checkout = get_random_string()
651+ target_dir = '~/' + get_random_string()
652 user = get_random_string()
653- return ('-r', repo, '--checkout-name', checkout, '-u', user)
654+ return (target_dir, '-u', user)
655
656
657 class FinishInitHostTest(StepsBasedSubCommandTestMixin, unittest.TestCase):
658
659 sub_command_class = finish_inithost.SubCommand
660 expected_arguments = get_arguments()
661- expected_handlers = (
662- handlers.handle_user,
663- handlers.handle_directories,
664- handlers.handle_code_dir,
665- )
666+ expected_handlers = (handlers.handle_user, handlers.handle_target_dir)
667
668 @property
669 def expected_steps(self):
670
671=== modified file 'lpsetup/tests/subcommands/test_inithost.py'
672--- lpsetup/tests/subcommands/test_inithost.py 2012-07-13 15:16:12 +0000
673+++ lpsetup/tests/subcommands/test_inithost.py 2012-07-20 15:13:48 +0000
674@@ -18,12 +18,12 @@
675 )
676
677
678-initialize_step = (
679- inithost.initialize, ['user', 'full_name', 'email', 'lpuser',
680+initialize_step = (inithost.initialize, ['user'])
681+setup_home_step = (
682+ inithost.setup_home, ['user', 'full_name', 'email', 'lpuser',
683 'private_key', 'public_key', 'valid_ssh_keys', 'ssh_key_path',
684 ])
685-initialize_lxc_step = (
686- inithost.initialize_lxc, [])
687+initialize_lxc_step = (inithost.initialize_lxc, [])
688 setup_apt_step = (inithost.setup_apt, [])
689
690
691@@ -52,6 +52,7 @@
692 )
693 expected_steps = (
694 initialize_step,
695+ setup_home_step,
696 initialize_lxc_step,
697 setup_apt_step,)
698 needs_root = True
699@@ -111,8 +112,8 @@
700 '/tmp/foo', 'Hello, world!', 'a+')
701
702
703-class EnsureSSHKeysTestCase(unittest.TestCase):
704- """Tests for inithost.ensure_ssh_keys()."""
705+class SetupSSHTestCase(unittest.TestCase):
706+ """Tests for inithost.setup_ssh()."""
707
708 def setUp(self):
709 temp_file = tempfile.NamedTemporaryFile()
710@@ -123,42 +124,42 @@
711 self.addCleanup(os.remove, self.temp_filename)
712 self.addCleanup(os.remove, self.temp_filename + ".pub")
713
714- def test_ensure_ssh_keys_writes_to_ssh_key_path(self):
715- # If inithost.ensure_ssh_keys() is told that the keys it has
716+ def test_setup_ssh_writes_to_ssh_key_path(self):
717+ # If inithost.setup_ssh() is told that the keys it has
718 # been passed are valid, it will write them out to the provided
719 # ssh_key_path.
720 public_key = "Public"
721 private_key = "Private"
722- inithost.ensure_ssh_keys(
723+ inithost.setup_ssh(
724 self.ssh_dir, private_key, public_key, True, self.temp_filename)
725 with open(self.temp_filename + '.pub', 'r') as pub_file:
726 self.assertIn(public_key, pub_file.read())
727 with open(self.temp_filename, 'r') as priv_file:
728 self.assertIn(private_key, priv_file.read())
729
730- def test_ensure_ssh_keys_generates_keys_if_not_passed(self):
731- # If SSH keys aren't passed to ensure_ssh_keys(), the function
732+ def test_setup_ssh_generates_keys_if_not_passed(self):
733+ # If SSH keys aren't passed to setup_ssh(), the function
734 # will generate some.
735- inithost.ensure_ssh_keys(
736+ inithost.setup_ssh(
737 self.ssh_dir, None, None, False, self.temp_filename)
738 self.assertTrue(os.path.exists(self.temp_filename))
739 self.assertTrue(os.path.exists(self.temp_filename + ".pub"))
740
741- def test_ensure_ssh_keys_generates_other_files(self):
742- # ensure_ssh_keys() also generates an authorized_keys file and a
743+ def test_setup_ssh_generates_other_files(self):
744+ # setup_ssh() also generates an authorized_keys file and a
745 # known_hosts file in the ssh dir.
746- inithost.ensure_ssh_keys(
747+ inithost.setup_ssh(
748 self.ssh_dir, None, None, False, self.temp_filename)
749 self.assertTrue(
750 os.path.exists(os.path.join(self.ssh_dir, 'authorized_keys')))
751 self.assertTrue(
752 os.path.exists(os.path.join(self.ssh_dir, 'known_hosts')))
753
754- def test_ensure_ssh_keys_creates_ssh_dir(self):
755- # If the ssh_dir passed to ensure_ssh_keys() doesn't exist, it
756+ def test_setup_ssh_creates_ssh_dir(self):
757+ # If the ssh_dir passed to setup_ssh() doesn't exist, it
758 # will be created.
759 shutil.rmtree(self.ssh_dir)
760- inithost.ensure_ssh_keys(
761+ inithost.setup_ssh(
762 self.ssh_dir, None, None, False, self.temp_filename)
763 self.assertTrue(os.path.exists(self.ssh_dir))
764 self.assertTrue(os.path.isdir(self.ssh_dir))
765
766=== modified file 'lpsetup/tests/subcommands/test_initlxc.py'
767--- lpsetup/tests/subcommands/test_initlxc.py 2012-07-16 21:12:21 +0000
768+++ lpsetup/tests/subcommands/test_initlxc.py 2012-07-20 15:13:48 +0000
769@@ -55,6 +55,7 @@
770 )
771 expected_steps = (
772 initialize_step,
773+ test_inithost.setup_home_step,
774 create_lxc_step,
775 start_lxc_step,
776 wait_for_lxc_step,
777
778=== modified file 'lpsetup/tests/subcommands/test_initrepo.py'
779--- lpsetup/tests/subcommands/test_initrepo.py 2012-07-17 10:46:49 +0000
780+++ lpsetup/tests/subcommands/test_initrepo.py 2012-07-20 15:13:48 +0000
781@@ -6,7 +6,6 @@
782
783 import os
784 import shutil
785-import subprocess
786 import tempfile
787 import unittest
788
789@@ -23,21 +22,13 @@
790 from lpsetup.tests.utils import (
791 create_test_branch,
792 get_random_string,
793+ lpuser,
794+ skip_if_no_lpuser,
795 StepsBasedSubCommandTestMixin,
796 )
797 from lpsetup.utils import ConfigParser
798
799
800-try:
801- lpuser = run('bzr', 'launchpad-login').strip()
802-except subprocess.CalledProcessError:
803- lpuser = None
804-# Create a decorator to skip tests if lp login is not set.
805-skip_if_no_lpuser = unittest.skipIf(
806- lpuser is None,
807- 'You need to set up a Launchpad login is needed to run this test.')
808-
809-
810 fetch_step = (initrepo.fetch,
811 ['source', 'repository', 'branch_name', 'checkout_name', 'no_checkout'])
812 setup_bzr_locations_step = (initrepo.setup_bzr_locations,
813
814=== modified file 'lpsetup/tests/subcommands/test_install_lxc.py'
815--- lpsetup/tests/subcommands/test_install_lxc.py 2012-07-19 17:48:40 +0000
816+++ lpsetup/tests/subcommands/test_install_lxc.py 2012-07-20 15:13:48 +0000
817@@ -23,19 +23,18 @@
818 init_repo_in_lxc_step = (
819 install_lxc.init_repo_in_lxc, [
820 'lxc_name', 'ssh_key_path', 'home_dir', 'user', 'source', 'use_http',
821- 'branch_name', 'checkout_name', 'repository',
822+ 'branch_name', 'checkout_name', 'repository', 'no_checkout',
823 ])
824
825 update_in_lxc_step = (
826 install_lxc.update_in_lxc, [
827 'lxc_name', 'ssh_key_path', 'home_dir', 'user', 'external_path',
828- 'use_http', 'checkout_name', 'repository',
829+ 'target_dir', 'use_http',
830 ])
831
832 finish_inithost_in_lxc_step = (
833 install_lxc.finish_inithost_in_lxc, [
834- 'lxc_name', 'ssh_key_path', 'home_dir', 'user', 'repository',
835- 'checkout_name',
836+ 'lxc_name', 'ssh_key_path', 'home_dir', 'user', 'target_dir',
837 ])
838
839
840@@ -53,8 +52,9 @@
841 expected_arguments = get_arguments()
842 expected_handlers = test_initlxc.InitLxcTest.expected_handlers + (
843 handlers.handle_directories,
844- handlers.handle_code_dir,
845+ handlers.handle_branch_and_checkout,
846 handlers.handle_source,
847+ handlers.handle_target_from_repository,
848 )
849 expected_steps = test_initlxc.InitLxcTest.expected_steps + (
850 create_scripts_step,
851
852=== modified file 'lpsetup/tests/subcommands/test_update.py'
853--- lpsetup/tests/subcommands/test_update.py 2012-07-19 17:48:40 +0000
854+++ lpsetup/tests/subcommands/test_update.py 2012-07-20 15:13:48 +0000
855@@ -15,18 +15,19 @@
856
857
858 def get_arguments():
859+ target_dir = '~/' + get_random_string()
860+ external_path = get_random_string()
861 return (
862- '--external-path', get_random_string(),
863+ target_dir,
864+ '--external-path', external_path,
865 '--use-http',
866- '--repository', get_random_string(),
867- '--checkout-name', get_random_string(),
868 )
869
870 init_dir_step = (
871- update.initialize_directories, ['code_dir', 'external_path'])
872+ update.initialize_directories, ['target_dir', 'external_path'])
873 update_dep_step = (
874- update.update_dependencies, ['code_dir', 'external_path', 'use_http'])
875-update_tree_step = (update.update_tree, ['code_dir'])
876+ update.update_dependencies, ['target_dir', 'external_path', 'use_http'])
877+update_tree_step = (update.update_tree, ['target_dir'])
878
879
880 class UpdateTest(StepsBasedSubCommandTestMixin, unittest.TestCase):
881@@ -34,11 +35,7 @@
882 sub_command_name = 'update'
883 sub_command_class = update.SubCommand
884 expected_arguments = get_arguments()
885- expected_handlers = (
886- handlers.handle_user,
887- handlers.handle_directories,
888- handlers.handle_code_dir,
889- )
890+ expected_handlers = (handlers.handle_user, handlers.handle_target_dir)
891 expected_steps = (
892 init_dir_step,
893 update_dep_step,
894
895=== modified file 'lpsetup/tests/test_handlers.py'
896--- lpsetup/tests/test_handlers.py 2012-07-17 10:28:05 +0000
897+++ lpsetup/tests/test_handlers.py 2012-07-20 15:13:48 +0000
898@@ -9,20 +9,31 @@
899 import getpass
900 import os
901 import pwd
902+import shutil
903+import tempfile
904 import unittest
905
906-from shelltoolbox import cd
907+from shelltoolbox import (
908+ cd,
909+ environ,
910+ )
911
912 from lpsetup.exceptions import ValidationError
913 from lpsetup.handlers import (
914 handle_branch_and_checkout,
915 handle_directories,
916 handle_lpuser_as_username,
917+ handle_lpuser_from_lplogin,
918 handle_ssh_keys,
919+ handle_target_from_repository,
920 handle_testing,
921 handle_user,
922 handle_userdata,
923 )
924+from lpsetup.tests.utils import (
925+ lpuser,
926+ skip_if_no_lpuser,
927+ )
928
929
930 class HandlersTestMixin(object):
931@@ -167,7 +178,7 @@
932 handle_directories(namespace)
933
934
935-class HandleLPUserTest(unittest.TestCase):
936+class HandleLPUserAsUsernameTest(unittest.TestCase):
937
938 def test_lpuser(self):
939 # If lpuser is not provided by namespace, the user name is used.
940@@ -177,6 +188,44 @@
941 self.assertEqual(username, namespace.lpuser)
942
943
944+class HandleLPUserFromLPLoginTest(HandlersTestMixin, unittest.TestCase):
945+
946+ def temp_home(self):
947+ home = tempfile.mkdtemp()
948+ self.addCleanup(shutil.rmtree, home)
949+ return environ(HOME=home)
950+
951+ def test_lpuser_in_namespace(self):
952+ # Ensure nothing happens if the lpuser exists in the namespace.
953+ value = 'myuser'
954+ namespace = argparse.Namespace(lpuser=value)
955+ handle_lpuser_from_lplogin(namespace)
956+ self.assertEqual(value, namespace.lpuser)
957+
958+ @skip_if_no_lpuser
959+ def test_lpuser_retrieval(self):
960+ # Ensure lpuser is correctly retrieved using bzr if it does not
961+ # exist in the namespace.
962+ namespace = argparse.Namespace()
963+ handle_lpuser_from_lplogin(namespace)
964+ self.assertEqual(lpuser, namespace.lpuser)
965+
966+ def test_no_lpuser(self):
967+ # The validation fails if lpuser is not retrievable.
968+ with self.temp_home():
969+ namespace = argparse.Namespace()
970+ with self.assertNotValid('launchpad-login'):
971+ handle_lpuser_from_lplogin(namespace)
972+
973+ def test_use_http(self):
974+ # Ensure lpuser is set to None if not retrievable and
975+ # *use_http* is True.
976+ with self.temp_home():
977+ namespace = argparse.Namespace(use_http=True)
978+ handle_lpuser_from_lplogin(namespace)
979+ self.assertIsNone(namespace.lpuser)
980+
981+
982 class HandleSSHKeysTest(HandlersTestMixin, unittest.TestCase):
983
984 home_dir = '/tmp/__does_not_exist__'
985@@ -228,6 +277,36 @@
986 handle_ssh_keys(namespace)
987
988
989+class HandleTargetFromRepositoryTest(unittest.TestCase):
990+
991+ branch_name = 'branch'
992+ checkout_name = 'checkout'
993+ repository = '/repository/'
994+
995+ def get_target_dir(self, no_checkout):
996+ namespace = argparse.Namespace(
997+ branch_name=self.branch_name,
998+ checkout_name=self.checkout_name,
999+ repository=self.repository,
1000+ no_checkout=no_checkout)
1001+ handle_target_from_repository(namespace)
1002+ return namespace.target_dir
1003+
1004+ def test_checkout(self):
1005+ # Ensure the target dir points to branch_name if
1006+ # lightweight checkout is not used.
1007+ expected = os.path.join(self.repository, self.checkout_name)
1008+ target_dir = self.get_target_dir(False)
1009+ self.assertEqual(expected, target_dir)
1010+
1011+ def test_no_checkout(self):
1012+ # Ensure the target dir points to checkout_name if
1013+ # lightweight checkout is used.
1014+ expected = os.path.join(self.repository, self.branch_name)
1015+ target_dir = self.get_target_dir(True)
1016+ self.assertEqual(expected, target_dir)
1017+
1018+
1019 class HandleTestingTest(unittest.TestCase):
1020
1021 ctx = {
1022
1023=== modified file 'lpsetup/tests/utils.py'
1024--- lpsetup/tests/utils.py 2012-07-17 10:26:46 +0000
1025+++ lpsetup/tests/utils.py 2012-07-20 15:13:48 +0000
1026@@ -21,6 +21,7 @@
1027 import string
1028 from StringIO import StringIO
1029 import sys
1030+import subprocess
1031 import tempfile
1032 import unittest
1033
1034@@ -113,6 +114,16 @@
1035 return ''.join(random.sample(string.ascii_letters, size))
1036
1037
1038+# Retrieve the current lpuser to be used in tests.
1039+try:
1040+ lpuser = run('bzr', 'launchpad-login').strip()
1041+except subprocess.CalledProcessError:
1042+ lpuser = None
1043+# Create a decorator to skip tests if lp login is not set.
1044+skip_if_no_lpuser = unittest.skipIf(
1045+ lpuser is None, 'You need to set up a Launchpad login to run this test.')
1046+
1047+
1048 class SubCommandTestMixin(object):
1049
1050 sub_command_class = examples.SubCommand

Subscribers

People subscribed via source and target branches

to all changes: