Merge lp:~terceiro/lava-dashboard-tool/absorbed-by-lava-tool into lp:lava-dashboard-tool

Proposed by Antonio Terceiro
Status: Merged
Approved by: Senthil Kumaran S
Approved revision: 160
Merged at revision: 160
Proposed branch: lp:~terceiro/lava-dashboard-tool/absorbed-by-lava-tool
Merge into: lp:lava-dashboard-tool
Diff against target: 1230 lines (+4/-1176)
6 files modified
lava_dashboard_tool/__init__.py (+0/-23)
lava_dashboard_tool/commands.py (+0/-981)
lava_dashboard_tool/main.py (+0/-37)
lava_dashboard_tool/tests/__init__.py (+0/-52)
lava_dashboard_tool/tests/test_commands.py (+0/-44)
setup.py (+4/-39)
To merge this branch: bzr merge lp:~terceiro/lava-dashboard-tool/absorbed-by-lava-tool
Reviewer Review Type Date Requested Status
Senthil Kumaran S Approve
Review via email: mp+160193@code.launchpad.net

Description of the change

To post a comment you must log in.
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Seems fine to me -- is this really needed though? Couldn't one just
remove lava-dashboard-tool from the manifest?

Revision history for this message
Antonio Terceiro (terceiro) wrote :

On Tue, Apr 23, 2013 at 12:58:22AM -0000, Michael Hudson-Doyle wrote:
> Seems fine to me -- is this really needed though? Couldn't one just
> remove lava-dashboard-tool from the manifest?

People have them installed with pypi and APT, and we need to provide a
clean upgrade path so that they don't get two packages providing
lava_dashboard_tool Python module at the same time. For this we need to
make a last release of the deprecated packages with no code at all that
depends on the package/version that provides that code.

--
Antonio Terceiro
Software Engineer - Linaro
http://www.linaro.org

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

+1

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== removed directory 'lava_dashboard_tool'
2=== removed file 'lava_dashboard_tool/__init__.py'
3--- lava_dashboard_tool/__init__.py 2012-03-22 18:38:32 +0000
4+++ lava_dashboard_tool/__init__.py 1970-01-01 00:00:00 +0000
5@@ -1,23 +0,0 @@
6-# Copyright (C) 2010,2011 Linaro Limited
7-#
8-# Author: Zygmunt Krynicki <zygmunt.krynicki@linaro.org>
9-#
10-# This file is part of lava-dashboard-tool.
11-#
12-# lava-dashboard-tool is free software: you can redistribute it and/or modify
13-# it under the terms of the GNU Lesser General Public License version 3
14-# as published by the Free Software Foundation
15-#
16-# lava-dashboard-tool is distributed in the hope that it will be useful,
17-# but WITHOUT ANY WARRANTY; without even the implied warranty of
18-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19-# GNU General Public License for more details.
20-#
21-# You should have received a copy of the GNU Lesser General Public License
22-# along with lava-dashboard-tool. If not, see <http://www.gnu.org/licenses/>.
23-
24-"""
25-Launch Control Tool package
26-"""
27-
28-__version__ = (0, 8, 0, "dev", 0)
29
30=== removed file 'lava_dashboard_tool/commands.py'
31--- lava_dashboard_tool/commands.py 2012-03-22 18:12:14 +0000
32+++ lava_dashboard_tool/commands.py 1970-01-01 00:00:00 +0000
33@@ -1,981 +0,0 @@
34-# Copyright (C) 2010,2011 Linaro Limited
35-#
36-# Author: Zygmunt Krynicki <zygmunt.krynicki@linaro.org>
37-#
38-# This file is part of lava-dashboard-tool.
39-#
40-# lava-dashboard-tool is free software: you can redistribute it and/or modify
41-# it under the terms of the GNU Lesser General Public License version 3
42-# as published by the Free Software Foundation
43-#
44-# lava-dashboard-tool is distributed in the hope that it will be useful,
45-# but WITHOUT ANY WARRANTY; without even the implied warranty of
46-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
47-# GNU General Public License for more details.
48-#
49-# You should have received a copy of the GNU Lesser General Public License
50-# along with lava-dashboard-tool. If not, see <http://www.gnu.org/licenses/>.
51-
52-"""
53-Module with command-line tool commands that interact with the dashboard
54-server. All commands listed here should have counterparts in the
55-launch_control.dashboard_app.xml_rpc package.
56-"""
57-
58-import argparse
59-import contextlib
60-import errno
61-import os
62-import re
63-import socket
64-import sys
65-import urllib
66-import urlparse
67-import xmlrpclib
68-
69-import simplejson
70-from json_schema_validator.extensions import datetime_extension
71-
72-from lava_tool.authtoken import AuthenticatingServerProxy, KeyringAuthBackend
73-from lava.tool.commands import ExperimentalCommandMixIn
74-from lava.tool.command import Command, CommandGroup
75-
76-
77-class dashboard(CommandGroup):
78- """
79- Commands for interacting with LAVA Dashboard
80- """
81-
82- namespace = "lava.dashboard.commands"
83-
84-
85-class InsufficientServerVersion(Exception):
86- """
87- Exception raised when server version that a command interacts with is too
88- old to support required features.
89- """
90- def __init__(self, server_version, required_version):
91- self.server_version = server_version
92- self.required_version = required_version
93-
94-
95-class DataSetRenderer(object):
96- """
97- Support class for rendering a table out of list of dictionaries.
98-
99- It supports several features that make printing tabular data easier.
100- * Automatic layout
101- * Custom column headers
102- * Custom cell formatting
103- * Custom table captions
104- * Custom column ordering
105- * Custom Column separators
106- * Custom dataset notification
107-
108- The primary method is render() which does all of the above. You
109- need to pass a dataset argument which is a list of dictionaries.
110- Each dictionary must have the same keys. In particular the first row
111- is used to determine columns.
112- """
113- def __init__(self, column_map=None, row_formatter=None, empty=None,
114- order=None, caption=None, separator=" ", header_separator=None):
115- if column_map is None:
116- column_map = {}
117- if row_formatter is None:
118- row_formatter = {}
119- if empty is None:
120- empty = "There is no data to display"
121- self.column_map = column_map
122- self.row_formatter = row_formatter
123- self.empty = empty
124- self.order = order
125- self.separator = separator
126- self.caption = caption
127- self.header_separator = header_separator
128-
129- def _analyze_dataset(self, dataset):
130- """
131- Determine the columns that will be displayed and the maximum
132- length of each of those columns.
133-
134- Returns a tuple (dataset, columms, maxlen) where columns is a
135- list of column names and maxlen is a dictionary mapping from
136- column name to maximum length of any value in the row or the
137- column header and the dataset is a copy of the dataset altered
138- as necessary.
139-
140- Some examples:
141-
142- First the dataset, an array of dictionaries
143- >>> dataset = [
144- ... {'a': 'shorter', 'bee': ''},
145- ... {'a': 'little longer', 'bee': 'b'}]
146-
147- Note that column 'bee' is actually three characters long as the
148- column name made it wider.
149- >>> dataset_out, columns, maxlen = DataSetRenderer(
150- ... )._analyze_dataset(dataset)
151-
152- Unless you format rows with a custom function the data is not altered.
153- >>> dataset_out is dataset
154- True
155-
156- Columns come out in sorted alphabetic order
157- >>> columns
158- ['a', 'bee']
159-
160- Maximum length determines the width of each column. Note that
161- the header affects the column width.
162- >>> maxlen
163- {'a': 13, 'bee': 3}
164-
165- You can constrain or reorder columns. In that case columns you
166- decided to ignore are simply left out of the output.
167- >>> dataset_out, columns, maxlen = DataSetRenderer(
168- ... order=['bee'])._analyze_dataset(dataset)
169- >>> columns
170- ['bee']
171- >>> maxlen
172- {'bee': 3}
173-
174- You can format values anyway you like:
175- >>> dataset_out, columns, maxlen = DataSetRenderer(row_formatter={
176- ... 'bee': lambda value: "%10s" % value}
177- ... )._analyze_dataset(dataset)
178-
179- Dataset is altered to take account of the row formatting
180- function. The original dataset argument is copied.
181- >>> dataset_out
182- [{'a': 'shorter', 'bee': ' '}, {'a': 'little longer', 'bee': ' b'}]
183- >>> dataset_out is not dataset
184- True
185-
186- Columns stay the same though:
187- >>> columns
188- ['a', 'bee']
189-
190- Note how formatting altered the width of the column 'bee'
191- >>> maxlen
192- {'a': 13, 'bee': 10}
193-
194- You can also format columns (with nice aliases).Note how
195- column 'bee' maximum width is now dominated by the long column
196- name:
197- >>> dataset_out, columns, maxlen = DataSetRenderer(column_map={
198- ... 'bee': "Column B"})._analyze_dataset(dataset)
199- >>> maxlen
200- {'a': 13, 'bee': 8}
201- """
202- if self.order:
203- columns = self.order
204- else:
205- columns = sorted(dataset[0].keys())
206- if self.row_formatter:
207- dataset_out = [dict(row) for row in dataset]
208- else:
209- dataset_out = dataset
210- for row in dataset_out:
211- for column in row:
212- if column in self.row_formatter:
213- row[column] = self.row_formatter[column](row[column])
214- maxlen = dict(
215- [(column, max(
216- len(self.column_map.get(column, column)),
217- max([
218- len(str(row[column])) for row in dataset_out])))
219- for column in columns])
220- return dataset_out, columns, maxlen
221-
222- def _render_header(self, dataset, columns, maxlen):
223- """
224- Render a header, possibly with a caption string
225-
226- Caption is controlled by the constructor.
227- >>> dataset = [
228- ... {'a': 'shorter', 'bee': ''},
229- ... {'a': 'little longer', 'bee': 'b'}]
230- >>> columns = ['a', 'bee']
231- >>> maxlen = {'a': 13, 'bee': 3}
232-
233- By default there is no caption, just column names:
234- >>> DataSetRenderer()._render_header(
235- ... dataset, columns, maxlen)
236- a bee
237-
238- If you enable the header separator then column names will be visually
239- separated from the first row of data.
240- >>> DataSetRenderer(header_separator=True)._render_header(
241- ... dataset, columns, maxlen)
242- a bee
243- -----------------
244-
245- If you provide a caption it gets rendered as a centered
246- underlined text before the data:
247- >>> DataSetRenderer(caption="Dataset")._render_header(
248- ... dataset, columns, maxlen)
249- Dataset
250- =================
251- a bee
252-
253- You can use both caption and header separator
254- >>> DataSetRenderer(caption="Dataset", header_separator=True)._render_header(
255- ... dataset, columns, maxlen)
256- Dataset
257- =================
258- a bee
259- -----------------
260-
261- Observe how the total length of the output horizontal line
262- depends on the separator! Also note the columns labels are
263- aligned to the center of the column
264- >>> DataSetRenderer(caption="Dataset", separator=" | ")._render_header(
265- ... dataset, columns, maxlen)
266- Dataset
267- ===================
268- a | bee
269- """
270- total_len = sum(maxlen.itervalues())
271- if len(columns):
272- total_len += len(self.separator) * (len(columns) - 1)
273- # Print the caption
274- if self.caption:
275- print "{0:^{1}}".format(self.caption, total_len)
276- print "=" * total_len
277- # Now print the coulum names
278- print self.separator.join([
279- "{0:^{1}}".format(self.column_map.get(column, column),
280- maxlen[column]) for column in columns])
281- # Finally print the header separator
282- if self.header_separator:
283- print "-" * total_len
284-
285- def _render_rows(self, dataset, columns, maxlen):
286- """
287- Render rows of the dataset.
288-
289- Each row is printed on one line using the maxlen argument to
290- determine correct column size. Text is aligned left.
291-
292- First the dataset, columns and maxlen as produced by
293- _analyze_dataset()
294- >>> dataset = [
295- ... {'a': 'shorter', 'bee': ''},
296- ... {'a': 'little longer', 'bee': 'b'}]
297- >>> columns = ['a', 'bee']
298- >>> maxlen = {'a': 13, 'bee': 3}
299-
300- Now a plain table. Note! To really understand this test
301- you should check out the length of the strings below. There
302- are two more spaces after 'b' in the second row
303- >>> DataSetRenderer()._render_rows(dataset, columns, maxlen)
304- shorter
305- little longer b
306- """
307- for row in dataset:
308- print self.separator.join([
309- "{0!s:{1}}".format(row[column], maxlen[column])
310- for column in columns])
311-
312- def _render_dataset(self, dataset):
313- """
314- Render the header followed by the rows of data.
315- """
316- dataset, columns, maxlen = self._analyze_dataset(dataset)
317- self._render_header(dataset, columns, maxlen)
318- self._render_rows(dataset, columns, maxlen)
319-
320- def _render_empty_dataset(self):
321- """
322- Render empty dataset.
323-
324- By default it just prints out a fixed sentence:
325- >>> DataSetRenderer()._render_empty_dataset()
326- There is no data to display
327-
328- This can be changed by passing an argument to the constructor
329- >>> DataSetRenderer(empty="there is no data")._render_empty_dataset()
330- there is no data
331- """
332- print self.empty
333-
334- def render(self, dataset):
335- if len(dataset) > 0:
336- self._render_dataset(dataset)
337- else:
338- self._render_empty_dataset()
339-
340-
341-class XMLRPCCommand(Command):
342- """
343- Abstract base class for commands that interact with dashboard server
344- over XML-RPC.
345-
346- The only difference is that you should implement invoke_remote()
347- instead of invoke(). The provided implementation catches several
348- socket and XML-RPC errors and prints a pretty error message.
349- """
350-
351- @staticmethod
352- def _construct_xml_rpc_url(url):
353- """
354- Construct URL to the XML-RPC service out of the given URL
355- """
356- parts = urlparse.urlsplit(url)
357- if not parts.path.endswith("/RPC2/"):
358- path = parts.path.rstrip("/") + "/xml-rpc/"
359- else:
360- path = parts.path
361- return urlparse.urlunsplit(
362- (parts.scheme, parts.netloc, path, "", ""))
363-
364- @staticmethod
365- def _strict_server_version(version):
366- """
367- Calculate strict server version (as defined by
368- distutils.version.StrictVersion). This works by discarding .candidate
369- and .dev release-levels.
370- >>> XMLRPCCommand._strict_server_version("0.4.0.candidate.5")
371- '0.4.0'
372- >>> XMLRPCCommand._strict_server_version("0.4.0.dev.126")
373- '0.4.0'
374- >>> XMLRPCCommand._strict_server_version("0.4.0.alpha.1")
375- '0.4.0a1'
376- >>> XMLRPCCommand._strict_server_version("0.4.0.beta.2")
377- '0.4.0b2'
378- """
379- try:
380- major, minor, micro, releaselevel, serial = version.split(".")
381- except ValueError:
382- raise ValueError(
383- ("version %r does not follow pattern "
384- "'major.minor.micro.releaselevel.serial'") % version)
385- if releaselevel in ["dev", "candidate", "final"]:
386- return "%s.%s.%s" % (major, minor, micro)
387- elif releaselevel == "alpha":
388- return "%s.%s.%sa%s" % (major, minor, micro, serial)
389- elif releaselevel == "beta":
390- return "%s.%s.%sb%s" % (major, minor, micro, serial)
391- else:
392- raise ValueError(
393- ("releaselevel %r is not one of 'final', 'alpha', 'beta', "
394- "'candidate' or 'final'") % releaselevel)
395-
396- def _check_server_version(self, server_obj, required_version):
397- """
398- Check that server object has is at least required_version.
399-
400- This method may raise InsufficientServerVersion.
401- """
402- from distutils.version import StrictVersion, LooseVersion
403- # For backwards compatibility the server reports
404- # major.minor.micro.releaselevel.serial which is not PEP-386 compliant
405- server_version = StrictVersion(
406- self._strict_server_version(server_obj.version()))
407- required_version = StrictVersion(required_version)
408- if server_version < required_version:
409- raise InsufficientServerVersion(server_version, required_version)
410-
411- def __init__(self, parser, args):
412- super(XMLRPCCommand, self).__init__(parser, args)
413- xml_rpc_url = self._construct_xml_rpc_url(self.args.dashboard_url)
414- self.server = AuthenticatingServerProxy(
415- xml_rpc_url,
416- verbose=args.verbose_xml_rpc,
417- allow_none=True,
418- use_datetime=True,
419- auth_backend=KeyringAuthBackend())
420-
421- def use_non_legacy_api_if_possible(self, name='server'):
422- # Legacy APIs are registered in top-level object, non-legacy APIs are
423- # prefixed with extension name.
424- if "dashboard.version" in getattr(self, name).system.listMethods():
425- setattr(self, name, getattr(self, name).dashboard)
426-
427- @classmethod
428- def register_arguments(cls, parser):
429- dashboard_group = parser.add_argument_group("dashboard specific arguments")
430- default_dashboard_url = os.getenv("DASHBOARD_URL")
431- if default_dashboard_url:
432- dashboard_group.add_argument("--dashboard-url",
433- metavar="URL", help="URL of your validation dashboard (currently %(default)s)",
434- default=default_dashboard_url)
435- else:
436- dashboard_group.add_argument("--dashboard-url", required=True,
437- metavar="URL", help="URL of your validation dashboard")
438- debug_group = parser.add_argument_group("debugging arguments")
439- debug_group.add_argument("--verbose-xml-rpc",
440- action="store_true", default=False,
441- help="Show XML-RPC data")
442- return dashboard_group
443-
444- @contextlib.contextmanager
445- def safety_net(self):
446- try:
447- yield
448- except socket.error as ex:
449- print >> sys.stderr, "Unable to connect to server at %s" % (
450- self.args.dashboard_url,)
451- # It seems that some errors are reported as -errno
452- # while others as +errno.
453- ex.errno = abs(ex.errno)
454- if ex.errno == errno.ECONNREFUSED:
455- print >> sys.stderr, "Connection was refused."
456- parts = urlparse.urlsplit(self.args.dashboard_url)
457- if parts.netloc == "localhost:8000":
458- print >> sys.stderr, "Perhaps the server is not running?"
459- elif ex.errno == errno.ENOENT:
460- print >> sys.stderr, "Unable to resolve address"
461- else:
462- print >> sys.stderr, "Socket %d: %s" % (ex.errno, ex.strerror)
463- except xmlrpclib.ProtocolError as ex:
464- print >> sys.stderr, "Unable to exchange XML-RPC message with dashboard server"
465- print >> sys.stderr, "HTTP error code: %d/%s" % (ex.errcode, ex.errmsg)
466- except xmlrpclib.Fault as ex:
467- self.handle_xmlrpc_fault(ex.faultCode, ex.faultString)
468- except InsufficientServerVersion as ex:
469- print >> sys.stderr, ("This command requires at least server version "
470- "%s, actual server version is %s" %
471- (ex.required_version, ex.server_version))
472-
473- def invoke(self):
474- with self.safety_net():
475- self.use_non_legacy_api_if_possible()
476- return self.invoke_remote()
477-
478- def handle_xmlrpc_fault(self, faultCode, faultString):
479- if faultCode == 500:
480- print >> sys.stderr, "Dashboard server has experienced internal error"
481- print >> sys.stderr, faultString
482- else:
483- print >> sys.stderr, "XML-RPC error %d: %s" % (faultCode, faultString)
484-
485- def invoke_remote(self):
486- raise NotImplementedError()
487-
488-
489-class server_version(XMLRPCCommand):
490- """
491- Display dashboard server version
492- """
493-
494- def invoke_remote(self):
495- print "Dashboard server version: %s" % (self.server.version(),)
496-
497-
498-class put(XMLRPCCommand):
499- """
500- Upload a bundle on the server
501- """
502-
503- @classmethod
504- def register_arguments(cls, parser):
505- super(put, cls).register_arguments(parser)
506- parser.add_argument("LOCAL",
507- type=argparse.FileType("rb"),
508- help="pathname on the local file system")
509- parser.add_argument("REMOTE",
510- default="/anonymous/", nargs='?',
511- help="pathname on the server")
512-
513- def invoke_remote(self):
514- content = self.args.LOCAL.read()
515- filename = self.args.LOCAL.name
516- pathname = self.args.REMOTE
517- content_sha1 = self.server.put(content, filename, pathname)
518- print "Stored as bundle {0}".format(content_sha1)
519-
520- def handle_xmlrpc_fault(self, faultCode, faultString):
521- if faultCode == 404:
522- print >> sys.stderr, "Bundle stream %s does not exist" % (
523- self.args.REMOTE)
524- elif faultCode == 409:
525- print >> sys.stderr, "You have already uploaded this bundle to the dashboard"
526- else:
527- super(put, self).handle_xmlrpc_fault(faultCode, faultString)
528-
529-
530-class get(XMLRPCCommand):
531- """
532- Download a bundle from the server
533- """
534-
535- @classmethod
536- def register_arguments(cls, parser):
537- super(get, cls).register_arguments(parser)
538- parser.add_argument("SHA1",
539- type=str,
540- help="SHA1 of the bundle to download")
541- parser.add_argument("--overwrite",
542- action="store_true",
543- help="Overwrite files on the local disk")
544- parser.add_argument("--output", "-o",
545- type=argparse.FileType("wb"),
546- default=None,
547- help="Alternate name of the output file")
548-
549- def invoke_remote(self):
550- response = self.server.get(self.args.SHA1)
551- if self.args.output is None:
552- filename = self.args.SHA1
553- if os.path.exists(filename) and not self.args.overwrite:
554- print >> sys.stderr, "File {filename!r} already exists".format(
555- filename=filename)
556- print >> sys.stderr, "You may pass --overwrite to write over it"
557- return -1
558- stream = open(filename, "wb")
559- else:
560- stream = self.args.output
561- filename = self.args.output.name
562- stream.write(response['content'])
563- print "Downloaded bundle {0} to file {1!r}".format(
564- self.args.SHA1, filename)
565-
566- def handle_xmlrpc_fault(self, faultCode, faultString):
567- if faultCode == 404:
568- print >> sys.stderr, "Bundle {sha1} does not exist".format(
569- sha1=self.args.SHA1)
570- else:
571- super(get, self).handle_xmlrpc_fault(faultCode, faultString)
572-
573-
574-class deserialize(XMLRPCCommand):
575- """
576- Deserialize a bundle on the server
577- """
578-
579- @classmethod
580- def register_arguments(cls, parser):
581- super(deserialize, cls).register_arguments(parser)
582- parser.add_argument("SHA1",
583- type=str,
584- help="SHA1 of the bundle to deserialize")
585-
586- def invoke_remote(self):
587- response = self.server.deserialize(self.args.SHA1)
588- print "Bundle {sha1} deserialized".format(
589- sha1=self.args.SHA1)
590-
591- def handle_xmlrpc_fault(self, faultCode, faultString):
592- if faultCode == 404:
593- print >> sys.stderr, "Bundle {sha1} does not exist".format(
594- sha1=self.args.SHA1)
595- elif faultCode == 409:
596- print >> sys.stderr, "Unable to deserialize bundle {sha1}".format(
597- sha1=self.args.SHA1)
598- print >> sys.stderr, faultString
599- else:
600- super(deserialize, self).handle_xmlrpc_fault(faultCode, faultString)
601-
602-
603-def _get_pretty_renderer(**kwargs):
604- if "separator" not in kwargs:
605- kwargs["separator"] = " | "
606- if "header_separator" not in kwargs:
607- kwargs["header_separator"] = True
608- return DataSetRenderer(**kwargs)
609-
610-
611-class streams(XMLRPCCommand):
612- """
613- Show streams you have access to
614- """
615-
616- renderer = _get_pretty_renderer(
617- order=('pathname', 'bundle_count', 'name'),
618- column_map={
619- 'pathname': 'Pathname',
620- 'bundle_count': 'Number of bundles',
621- 'name': 'Name'},
622- row_formatter={
623- 'name': lambda name: name or "(not set)"},
624- empty="There are no streams you can access on the server",
625- caption="Bundle streams")
626-
627- def invoke_remote(self):
628- self.renderer.render(self.server.streams())
629-
630-
631-class bundles(XMLRPCCommand):
632- """
633- Show bundles in the specified stream
634- """
635-
636- renderer = _get_pretty_renderer(
637- column_map={
638- 'uploaded_by': 'Uploader',
639- 'uploaded_on': 'Upload date',
640- 'content_filename': 'File name',
641- 'content_sha1': 'SHA1',
642- 'is_deserialized': "Deserialized?"},
643- row_formatter={
644- 'is_deserialized': lambda x: "yes" if x else "no",
645- 'uploaded_by': lambda x: x or "(anonymous)",
646- 'uploaded_on': lambda x: x.strftime("%Y-%m-%d %H:%M:%S")},
647- order=('content_sha1', 'content_filename', 'uploaded_by',
648- 'uploaded_on', 'is_deserialized'),
649- empty="There are no bundles in this stream",
650- caption="Bundles",
651- separator=" | ")
652-
653- @classmethod
654- def register_arguments(cls, parser):
655- super(bundles, cls).register_arguments(parser)
656- parser.add_argument("PATHNAME",
657- default="/anonymous/", nargs='?',
658- help="pathname on the server (defaults to %(default)s)")
659-
660- def invoke_remote(self):
661- self.renderer.render(self.server.bundles(self.args.PATHNAME))
662-
663- def handle_xmlrpc_fault(self, faultCode, faultString):
664- if faultCode == 404:
665- print >> sys.stderr, "Bundle stream %s does not exist" % (
666- self.args.PATHNAME)
667- else:
668- super(bundles, self).handle_xmlrpc_fault(faultCode, faultString)
669-
670-
671-class make_stream(XMLRPCCommand):
672- """
673- Create a bundle stream on the server
674- """
675-
676- @classmethod
677- def register_arguments(cls, parser):
678- super(make_stream, cls).register_arguments(parser)
679- parser.add_argument(
680- "pathname",
681- type=str,
682- help="Pathname of the bundle stream to create")
683- parser.add_argument(
684- "--name",
685- type=str,
686- default="",
687- help="Name of the bundle stream (description)")
688-
689- def invoke_remote(self):
690- self._check_server_version(self.server, "0.3")
691- pathname = self.server.make_stream(self.args.pathname, self.args.name)
692- print "Bundle stream {pathname} created".format(pathname=pathname)
693-
694-
695-class backup(XMLRPCCommand):
696- """
697- Backup data uploaded to a dashboard instance.
698-
699
700- Not all data is preserved. The following data is lost: identity of the user
701- that uploaded each bundle, time of uploading and deserialization on the
702- server, name of the bundle stream that contained the data
703- """
704-
705- @classmethod
706- def register_arguments(cls, parser):
707- super(backup, cls).register_arguments(parser)
708- parser.add_argument("BACKUP_DIR", type=str,
709- help="Directory to backup to")
710-
711- def invoke_remote(self):
712- if not os.path.exists(self.args.BACKUP_DIR):
713- os.mkdir(self.args.BACKUP_DIR)
714- for bundle_stream in self.server.streams():
715- print "Processing stream %s" % bundle_stream["pathname"]
716- bundle_stream_dir = os.path.join(self.args.BACKUP_DIR, urllib.quote_plus(bundle_stream["pathname"]))
717- if not os.path.exists(bundle_stream_dir):
718- os.mkdir(bundle_stream_dir)
719- with open(os.path.join(bundle_stream_dir, "metadata.json"), "wt") as stream:
720- simplejson.dump({
721- "pathname": bundle_stream["pathname"],
722- "name": bundle_stream["name"],
723- "user": bundle_stream["user"],
724- "group": bundle_stream["group"],
725- }, stream)
726- for bundle in self.server.bundles(bundle_stream["pathname"]):
727- print " * Backing up bundle %s" % bundle["content_sha1"]
728- data = self.server.get(bundle["content_sha1"])
729- bundle_pathname = os.path.join(bundle_stream_dir, bundle["content_sha1"])
730- # Note: we write bundles as binary data to preserve anything the user might have dumped on us
731- with open(bundle_pathname + ".json", "wb") as stream:
732- stream.write(data["content"])
733- with open(bundle_pathname + ".metadata.json", "wt") as stream:
734- simplejson.dump({
735- "uploaded_by": bundle["uploaded_by"],
736- "uploaded_on": datetime_extension.to_json(bundle["uploaded_on"]),
737- "content_filename": bundle["content_filename"],
738- "content_sha1": bundle["content_sha1"],
739- "content_size": bundle["content_size"],
740- }, stream)
741-
742-
743-class restore(XMLRPCCommand):
744- """
745- Restore a dashboard instance from backup
746- """
747-
748- @classmethod
749- def register_arguments(cls, parser):
750- super(restore, cls).register_arguments(parser)
751- parser.add_argument("BACKUP_DIR", type=str,
752- help="Directory to backup from")
753-
754- def invoke_remote(self):
755- self._check_server_version(self.server, "0.3")
756- for stream_pathname_quoted in os.listdir(self.args.BACKUP_DIR):
757- filesystem_stream_pathname = os.path.join(self.args.BACKUP_DIR, stream_pathname_quoted)
758- if not os.path.isdir(filesystem_stream_pathname):
759- continue
760- stream_pathname = urllib.unquote(stream_pathname_quoted)
761- if os.path.exists(os.path.join(filesystem_stream_pathname, "metadata.json")):
762- with open(os.path.join(filesystem_stream_pathname, "metadata.json"), "rt") as stream:
763- stream_metadata = simplejson.load(stream)
764- else:
765- stream_metadata = {}
766- print "Processing stream %s" % stream_pathname
767- try:
768- self.server.make_stream(stream_pathname, stream_metadata.get("name", "Restored from backup"))
769- except xmlrpclib.Fault as ex:
770- if ex.faultCode != 409:
771- raise
772- for content_sha1 in [item[:-len(".json")] for item in os.listdir(filesystem_stream_pathname) if item.endswith(".json") and not item.endswith(".metadata.json") and item != "metadata.json"]:
773- filesystem_content_filename = os.path.join(filesystem_stream_pathname, content_sha1 + ".json")
774- if not os.path.isfile(filesystem_content_filename):
775- continue
776- with open(os.path.join(filesystem_stream_pathname, content_sha1) + ".metadata.json", "rt") as stream:
777- bundle_metadata = simplejson.load(stream)
778- with open(filesystem_content_filename, "rb") as stream:
779- content = stream.read()
780- print " * Restoring bundle %s" % content_sha1
781- try:
782- self.server.put(content, bundle_metadata["content_filename"], stream_pathname)
783- except xmlrpclib.Fault as ex:
784- if ex.faultCode != 409:
785- raise
786-
787-
788-class pull(ExperimentalCommandMixIn, XMLRPCCommand):
789- """
790- Copy bundles and bundle streams from one dashboard to another.
791-
792
793- This command checks for two environment varialbes:
794- The value of DASHBOARD_URL is used as a replacement for --dashbard-url.
795- The value of REMOTE_DASHBOARD_URL as a replacement for FROM.
796- Their presence automatically makes the corresponding argument optional.
797- """
798-
799- def __init__(self, parser, args):
800- super(pull, self).__init__(parser, args)
801- remote_xml_rpc_url = self._construct_xml_rpc_url(self.args.FROM)
802- self.remote_server = AuthenticatingServerProxy(
803- remote_xml_rpc_url,
804- verbose=args.verbose_xml_rpc,
805- use_datetime=True,
806- allow_none=True,
807- auth_backend=KeyringAuthBackend())
808- self.use_non_legacy_api_if_possible('remote_server')
809-
810- @classmethod
811- def register_arguments(cls, parser):
812- group = super(pull, cls).register_arguments(parser)
813- default_remote_dashboard_url = os.getenv("REMOTE_DASHBOARD_URL")
814- if default_remote_dashboard_url:
815- group.add_argument(
816- "FROM", nargs="?",
817- help="URL of the remote validation dashboard (currently %(default)s)",
818- default=default_remote_dashboard_url)
819- else:
820- group.add_argument(
821- "FROM",
822- help="URL of the remote validation dashboard)")
823- group.add_argument("STREAM", nargs="*", help="Streams to pull from (all by default)")
824-
825- @staticmethod
826- def _filesizeformat(num_bytes):
827- """
828- Formats the value like a 'human-readable' file size (i.e. 13 KB, 4.1 MB,
829- 102 num_bytes, etc).
830- """
831- try:
832- num_bytes = float(num_bytes)
833- except (TypeError, ValueError, UnicodeDecodeError):
834- return "%(size)d byte", "%(size)d num_bytes" % {'size': 0}
835-
836- filesize_number_format = lambda value: "%0.2f" % (round(value, 1),)
837-
838- if num_bytes < 1024:
839- return "%(size)d bytes" % {'size': num_bytes}
840- if num_bytes < 1024 * 1024:
841- return "%s KB" % filesize_number_format(num_bytes / 1024)
842- if num_bytes < 1024 * 1024 * 1024:
843- return "%s MB" % filesize_number_format(num_bytes / (1024 * 1024))
844- return "%s GB" % filesize_number_format(num_bytes / (1024 * 1024 * 1024))
845-
846- def invoke_remote(self):
847- self._check_server_version(self.server, "0.3")
848-
849- print "Checking local and remote streams"
850- remote = self.remote_server.streams()
851- if self.args.STREAM:
852- # Check that all requested streams are available remotely
853- requested_set = frozenset(self.args.STREAM)
854- remote_set = frozenset((stream["pathname"] for stream in remote))
855- unavailable_set = requested_set - remote_set
856- if unavailable_set:
857- print >> sys.stderr, "Remote stream not found: %s" % ", ".join(unavailable_set)
858- return -1
859- # Limit to requested streams if necessary
860- remote = [stream for stream in remote if stream["pathname"] in requested_set]
861- local = self.server.streams()
862- missing_pathnames = set([stream["pathname"] for stream in remote]) - set([stream["pathname"] for stream in local])
863- for stream in remote:
864- if stream["pathname"] in missing_pathnames:
865- self.server.make_stream(stream["pathname"], stream["name"])
866- local_bundles = []
867- else:
868- local_bundles = [bundle for bundle in self.server.bundles(stream["pathname"])]
869- remote_bundles = [bundle for bundle in self.remote_server.bundles(stream["pathname"])]
870- missing_bundles = set((bundle["content_sha1"] for bundle in remote_bundles))
871- missing_bundles -= set((bundle["content_sha1"] for bundle in local_bundles))
872- try:
873- missing_bytes = sum(
874- (bundle["content_size"]
875- for bundle in remote_bundles
876- if bundle["content_sha1"] in missing_bundles))
877- except KeyError as ex:
878- # Older servers did not return content_size so this part is optional
879- missing_bytes = None
880- if missing_bytes:
881- print "Stream %s needs update (%s)" % (stream["pathname"], self._filesizeformat(missing_bytes))
882- elif missing_bundles:
883- print "Stream %s needs update (no estimate available)" % (stream["pathname"],)
884- else:
885- print "Stream %s is up to date" % (stream["pathname"],)
886- for content_sha1 in missing_bundles:
887- print "Getting %s" % (content_sha1,),
888- sys.stdout.flush()
889- data = self.remote_server.get(content_sha1)
890- print "got %s, storing" % (self._filesizeformat(len(data["content"]))),
891- sys.stdout.flush()
892- try:
893- self.server.put(data["content"], data["content_filename"], stream["pathname"])
894- except xmlrpclib.Fault as ex:
895- if ex.faultCode == 409: # duplicate
896- print "already present (in another stream)"
897- else:
898- raise
899- else:
900- print "done"
901-
902-
903-class data_views(ExperimentalCommandMixIn, XMLRPCCommand):
904- """
905- Show data views defined on the server
906- """
907- renderer = _get_pretty_renderer(
908- column_map={
909- 'name': 'Name',
910- 'summary': 'Summary',
911- },
912- order=('name', 'summary'),
913- empty="There are no data views defined yet",
914- caption="Data Views")
915-
916- def invoke_remote(self):
917- self._check_server_version(self.server, "0.4")
918- self.renderer.render(self.server.data_views())
919- print
920- print "Tip: to invoke a data view try `lc-tool query-data-view`"
921-
922-
923-class query_data_view(ExperimentalCommandMixIn, XMLRPCCommand):
924- """
925- Invoke a specified data view
926- """
927- @classmethod
928- def register_arguments(cls, parser):
929- super(query_data_view, cls).register_arguments(parser)
930- parser.add_argument("QUERY", metavar="QUERY", nargs="...",
931- help="Data view name and any optional and required arguments")
932-
933- def _probe_data_views(self):
934- """
935- Probe the server for information about data views
936- """
937- with self.safety_net():
938- self.use_non_legacy_api_if_possible()
939- self._check_server_version(self.server, "0.4")
940- return self.server.data_views()
941-
942- def reparse_arguments(self, parser, raw_args):
943- self.data_views = self._probe_data_views()
944- if self.data_views is None:
945- return
946- # Here we hack a little, the last actuin is the QUERY action added
947- # in register_arguments above. By removing it we make the output
948- # of lc-tool query-data-view NAME --help more consistent.
949- del parser._actions[-1]
950- subparsers = parser.add_subparsers(
951- title="Data views available on the server")
952- for data_view in self.data_views:
953- data_view_parser = subparsers.add_parser(
954- data_view["name"],
955- help=data_view["summary"],
956- epilog=data_view["documentation"])
957- data_view_parser.set_defaults(data_view=data_view)
958- group = data_view_parser.add_argument_group("Data view parameters")
959- for argument in data_view["arguments"]:
960- if argument["default"] is None:
961- group.add_argument(
962- "--{name}".format(name=argument["name"].replace("_", "-")),
963- dest=argument["name"],
964- help=argument["help"],
965- type=str,
966- required=True)
967- else:
968- group.add_argument(
969- "--{name}".format(name=argument["name"].replace("_", "-")),
970- dest=argument["name"],
971- help=argument["help"],
972- type=str,
973- default=argument["default"])
974- self.args = self.parser.parse_args(raw_args)
975-
976- def invoke(self):
977- # Override and _not_ call 'use_non_legacy_api_if_possible' as we
978- # already did this reparse_arguments
979- with self.safety_net():
980- return self.invoke_remote()
981-
982- def invoke_remote(self):
983- if self.data_views is None:
984- return -1
985- self._check_server_version(self.server, "0.4")
986- # Build a collection of arguments for data view
987- data_view_args = {}
988- for argument in self.args.data_view["arguments"]:
989- arg_name = argument["name"]
990- if arg_name in self.args:
991- data_view_args[arg_name] = getattr(self.args, arg_name)
992- # Invoke the data view
993- response = self.server.query_data_view(self.args.data_view["name"], data_view_args)
994- # Create a pretty-printer
995- renderer = _get_pretty_renderer(
996- caption=self.args.data_view["summary"],
997- order=[item["name"] for item in response["columns"]])
998- # Post-process the data so that it fits the printer
999- data_for_renderer = [
1000- dict(zip(
1001- [column["name"] for column in response["columns"]],
1002- row))
1003- for row in response["rows"]]
1004- # Print the data
1005- renderer.render(data_for_renderer)
1006-
1007-
1008-class version(Command):
1009- """
1010- Show dashboard client version
1011- """
1012- def invoke(self):
1013- import versiontools
1014- from lava_dashboard_tool import __version__
1015- print "Dashboard client version: {version}".format(
1016- version=versiontools.format_version(__version__))
1017
1018=== removed file 'lava_dashboard_tool/main.py'
1019--- lava_dashboard_tool/main.py 2011-06-23 11:23:24 +0000
1020+++ lava_dashboard_tool/main.py 1970-01-01 00:00:00 +0000
1021@@ -1,37 +0,0 @@
1022-# Copyright (C) 2011 Linaro Limited
1023-#
1024-# Author: Zygmunt Krynicki <zygmunt.krynicki@linaro.org>
1025-# Author: Michael Hudson-Doyle <michael.hudson@linaro.org>
1026-#
1027-# This file is part of lava-dashboard-tool.
1028-#
1029-# lava-dashboard-tool is free software: you can redistribute it and/or modify
1030-# it under the terms of the GNU Lesser General Public License version 3
1031-# as published by the Free Software Foundation
1032-#
1033-# lava-dashboard-tool is distributed in the hope that it will be useful,
1034-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1035-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1036-# GNU General Public License for more details.
1037-#
1038-# You should have received a copy of the GNU Lesser General Public License
1039-# along with lava-dashboard-tool. If not, see <http://www.gnu.org/licenses/>.
1040-
1041-
1042-from lava_tool.dispatcher import LavaDispatcher, run_with_dispatcher_class
1043-
1044-
1045-class LaunchControlDispatcher(LavaDispatcher):
1046-
1047- toolname = 'lava_dashboard_tool'
1048- description = """
1049- Command line tool for interacting with Launch Control
1050- """
1051- epilog = """
1052- Please report all bugs using the Launchpad bug tracker:
1053- http://bugs.launchpad.net/lava-dashboard-tool/+filebug
1054- """
1055-
1056-
1057-def main():
1058- run_with_dispatcher_class(LaunchControlDispatcher)
1059
1060=== removed directory 'lava_dashboard_tool/tests'
1061=== removed file 'lava_dashboard_tool/tests/__init__.py'
1062--- lava_dashboard_tool/tests/__init__.py 2011-06-23 11:23:24 +0000
1063+++ lava_dashboard_tool/tests/__init__.py 1970-01-01 00:00:00 +0000
1064@@ -1,52 +0,0 @@
1065-# Copyright (C) 2010,2011 Linaro Limited
1066-#
1067-# Author: Zygmunt Krynicki <zygmunt.krynicki@linaro.org>
1068-#
1069-# This file is part of lava-dashboard-tool.
1070-#
1071-# lava-dashboard-tool is free software: you can redistribute it and/or modify
1072-# it under the terms of the GNU Lesser General Public License version 3
1073-# as published by the Free Software Foundation
1074-#
1075-# lava-dashboard-tool is distributed in the hope that it will be useful,
1076-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1077-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1078-# GNU General Public License for more details.
1079-#
1080-# You should have received a copy of the GNU Lesser General Public License
1081-# along with lava-dashboard-tool. If not, see <http://www.gnu.org/licenses/>.
1082-
1083-"""
1084-Package with unit tests for lava_dashboard_tool
1085-"""
1086-
1087-import doctest
1088-import unittest
1089-
1090-
1091-def app_modules():
1092- return [
1093- 'lava_dashboard_tool.commands',
1094- ]
1095-
1096-
1097-def test_modules():
1098- return [
1099- 'lava_dashboard_tool.tests.test_commands',
1100- ]
1101-
1102-
1103-def test_suite():
1104- """
1105- Build an unittest.TestSuite() object with all the tests in _modules.
1106- Each module is harvested for both regular unittests and doctests
1107- """
1108- modules = app_modules() + test_modules()
1109- suite = unittest.TestSuite()
1110- loader = unittest.TestLoader()
1111- for name in modules:
1112- unit_suite = loader.loadTestsFromName(name)
1113- suite.addTests(unit_suite)
1114- doc_suite = doctest.DocTestSuite(name)
1115- suite.addTests(doc_suite)
1116- return suite
1117
1118=== removed file 'lava_dashboard_tool/tests/test_commands.py'
1119--- lava_dashboard_tool/tests/test_commands.py 2011-06-23 11:23:24 +0000
1120+++ lava_dashboard_tool/tests/test_commands.py 1970-01-01 00:00:00 +0000
1121@@ -1,44 +0,0 @@
1122-# Copyright (C) 2010,2011 Linaro Limited
1123-#
1124-# Author: Zygmunt Krynicki <zygmunt.krynicki@linaro.org>
1125-#
1126-# This file is part of lava-dashboard-tool.
1127-#
1128-# lava-dashboard-tool is free software: you can redistribute it and/or modify
1129-# it under the terms of the GNU Lesser General Public License version 3
1130-# as published by the Free Software Foundation
1131-#
1132-# lava-dashboard-tool is distributed in the hope that it will be useful,
1133-# but WITHOUT ANY WARRANTY; without even the implied warranty of
1134-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1135-# GNU General Public License for more details.
1136-#
1137-# You should have received a copy of the GNU Lesser General Public License
1138-# along with lava-dashboard-tool. If not, see <http://www.gnu.org/licenses/>.
1139-
1140-"""
1141-Unit tests for the launch_control.commands package
1142-"""
1143-
1144-from unittest import TestCase
1145-
1146-from lava_dashboard_tool.commands import XMLRPCCommand
1147-
1148-
1149-class XMLRPCCommandTestCase(TestCase):
1150-
1151- def test_construct_xml_rpc_url_preserves_path(self):
1152- self.assertEqual(
1153- XMLRPCCommand._construct_xml_rpc_url("http://domain/path"),
1154- "http://domain/path/xml-rpc/")
1155- self.assertEqual(
1156- XMLRPCCommand._construct_xml_rpc_url("http://domain/path/"),
1157- "http://domain/path/xml-rpc/")
1158-
1159- def test_construct_xml_rpc_url_adds_proper_suffix(self):
1160- self.assertEqual(
1161- XMLRPCCommand._construct_xml_rpc_url("http://domain/"),
1162- "http://domain/xml-rpc/")
1163- self.assertEqual(
1164- XMLRPCCommand._construct_xml_rpc_url("http://domain"),
1165- "http://domain/xml-rpc/")
1166
1167=== modified file 'setup.py'
1168--- setup.py 2012-03-22 18:12:14 +0000
1169+++ setup.py 2013-04-22 19:30:37 +0000
1170@@ -23,48 +23,14 @@
1171
1172 setup(
1173 name='lava-dashboard-tool',
1174- version=":versiontools:lava_dashboard_tool:__version__",
1175+ version="0.8",
1176 author="Zygmunt Krynicki",
1177 author_email="zygmunt.krynicki@linaro.org",
1178 packages=find_packages(),
1179- description="Command line utility for Launch Control",
1180+ description="Command line utility for Launch Control (deprecated)",
1181 url='https://launchpad.net/lava-dashboard-tool',
1182 test_suite='lava_dashboard_tool.tests.test_suite',
1183 license="LGPLv3",
1184- entry_points="""
1185- [console_scripts]
1186- lava-dashboard-tool=lava_dashboard_tool.main:main
1187- [lava.commands]
1188- dashboard=lava_dashboard_tool.commands:dashboard
1189- [lava.dashboard.commands]
1190- backup=lava_dashboard_tool.commands:backup
1191- bundles=lava_dashboard_tool.commands:bundles
1192- data_views=lava_dashboard_tool.commands:data_views
1193- deserialize=lava_dashboard_tool.commands:deserialize
1194- get=lava_dashboard_tool.commands:get
1195- make_stream=lava_dashboard_tool.commands:make_stream
1196- pull=lava_dashboard_tool.commands:pull
1197- put=lava_dashboard_tool.commands:put
1198- query_data_view=lava_dashboard_tool.commands:query_data_view
1199- restore=lava_dashboard_tool.commands:restore
1200- server_version=lava_dashboard_tool.commands:server_version
1201- streams=lava_dashboard_tool.commands:streams
1202- version=lava_dashboard_tool.commands:version
1203- [lava_dashboard_tool.commands]
1204- backup=lava_dashboard_tool.commands:backup
1205- bundles=lava_dashboard_tool.commands:bundles
1206- data_views=lava_dashboard_tool.commands:data_views
1207- deserialize=lava_dashboard_tool.commands:deserialize
1208- get=lava_dashboard_tool.commands:get
1209- make_stream=lava_dashboard_tool.commands:make_stream
1210- pull=lava_dashboard_tool.commands:pull
1211- put=lava_dashboard_tool.commands:put
1212- query_data_view=lava_dashboard_tool.commands:query_data_view
1213- restore=lava_dashboard_tool.commands:restore
1214- server_version=lava_dashboard_tool.commands:server_version
1215- streams=lava_dashboard_tool.commands:streams
1216- version=lava_dashboard_tool.commands:version
1217- """,
1218 classifiers=[
1219 "Development Status :: 4 - Beta",
1220 "Intended Audience :: Developers",
1221@@ -75,9 +41,8 @@
1222 "Programming Language :: Python :: 2.7",
1223 "Topic :: Software Development :: Testing"],
1224 install_requires=[
1225- 'lava-tool [auth] >= 0.4',
1226- 'json-schema-validator >= 2.0',
1227- 'versiontools >= 1.3.1'],
1228+ 'lava-tool >= 0.7.dev',
1229+ ],
1230 setup_requires=['versiontools >= 1.3.1'],
1231 tests_require=[],
1232 zip_safe=True)

Subscribers

People subscribed via source and target branches