Merge lp:~canonical-platform-qa/qakit/performance_solution into lp:qakit

[Sergio Cazzolato]

This branch involves many changes in order to extend the performance features that was previously used just for app startup, supporting now different kind of times based tests such as the scalability ones. The change includes:
. Adding scalability to qakit
. html report removed
. Generic report generator created in the engine dir
. Dashboard updated
. configuration moved to config file (see README)
. readme file updated

[lp:~canonical-platform-qa/qakit/performance_solution]

56. By Sergio Cazzolato on 2016-03-16

Simplify the settings config for the eve service

57. By Sergio Cazzolato on 2016-03-16

Adding missing files and updating README

58. By Sergio Cazzolato on 2016-03-21

Editing the scalability names used in the README

59. By Sergio Cazzolato on 2016-03-23

Fix based on error doing requests.post in jenkins due to an old version installed there

60. By Sergio Cazzolato on 2016-03-29

Force matplotlib to not use any Xwindows backend

61. By Sergio Cazzolato on 2016-03-29

Force matplotlib to not use any Xwindows backend

62. By Sergio Cazzolato on 2016-03-30

Minor fixes on dashboard index js

63. By Sergio Cazzolato on 2016-03-30

Minor fixes on dashboard index js

64. By Sergio Cazzolato on 2016-03-30

Using double quotes for jquery locators

65. By Sergio Cazzolato on 2016-03-30

Updating js to create pdf

66. By Sergio Cazzolato on 2016-04-13

Adding support for perfomance tests and improving plotting sticks

67. By Sergio Cazzolato on 2016-04-13

minor change in css to center performance graphs

68. By Sergio Cazzolato on 2016-04-14

Changing name of report

69. By Sergio Cazzolato on 2016-04-15

Adding missing files

70. By Sergio Cazzolato on 2016-04-15

Fix unintentional deletion

71. By Sergio Cazzolato on 2016-04-22

Allow to join reports

72. By Sergio Cazzolato on 2016-04-22

Updating config values for performance

73. By Sergio Cazzolato on 2016-04-22

Fix reports path dir

=== modified file 'README'
--- README	2016-02-18 14:19:19 +0000
+++ README	2016-04-22 20:41:08 +0000
@@ -88,11 +88,24 @@
     [APP_STARTUP]
     EVE_HOST = [dest_ip]
     EVE_PORT = [dest_port]
-    EVE_PATH = appstartup
-    HTML_REPORT = True
+    EVE_PATH = perf
     JSON_REPORT = True
     BUILDS_TO_PLOT = 10
 
+    [APP_STARTUP]
+    ALLOWED_APPS = webbrowser, address_book, calculator, here, dialer, clock, messaging, ebay, camera, system_settings, music, gallery
+    TYPES = cold, hot
+    TITLES = Cold Start, Hot Start
+    COLORS = 3366FF, FF1A00
+    ECOLORS = FF1A00, 3366FF
+
+    [SCALABILITY]
+    ALLOWED_NAMES = pictures_processed, pictures_gallery, my_pictures, pictures_carousel
+    LOADS = 1024, 2048, 4096, 8192, 16384
+    TITLES = Load 1024, Load 2048, Load 4096, Load 8192, Load 16384
+    COLORS = FFBF00, 81F7D8, 9F81F7, F781BE, 0040FF
+    ECOLORS = 0040FF, F781BE, 9F81F7, 81F7D8, FFBF00
+
 Having a test-results.subunit file, run as following: python3 qakit/appstartup/orchestrator.py pathto/test-results.subunit
 
 HTML and json reports are generated in the subunit directory. All the pictures will be stored in the appstartup_data dir.

=== removed directory 'qakit/appstartup'
=== removed file 'qakit/appstartup/__init__.py'
--- qakit/appstartup/__init__.py	2016-01-22 20:54:59 +0000
+++ qakit/appstartup/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,33 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import configparser
-
-import qakit.config as qakit_config
-
-allowed_apps = ['webbrowser', 'address_book', 'calculator', 'here', 'dialer',
-                'clock', 'messaging', 'ebay', 'camera', 'system_settings',
-                'music', 'gallery']
-
-
-def _read_config(config_filepath):
-    """ Parse the config at the given filepath and return it """
-    config_file = configparser.ConfigParser()
-    config_file.read(config_filepath)
-    return config_file
-
-config_dict = _read_config(qakit_config.get_config_file_location())

=== removed file 'qakit/appstartup/dao.py'
--- qakit/appstartup/dao.py	2016-02-18 14:19:19 +0000
+++ qakit/appstartup/dao.py	1970-01-01 00:00:00 +0000
@@ -1,102 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import logging
-import requests
-
-from qakit.appstartup import config_dict as config
-
-BASE_URL = 'http://{}:{}/{}'.format(config['APP_STARTUP']['EVE_HOST'],
-                                    config['APP_STARTUP']['EVE_PORT'],
-                                    config['APP_STARTUP']['EVE_PATH'])
-
-logger = logging.getLogger(__name__)
-
-
-def get_headers():
-    return {'Content-Type': 'application/json'}
-
-
-def _get_where_param(**params):
-    """ Get a where parameter used to filter when getting a query to mongo
-    :param params: A dict with all the parameters used to filter
-    """
-    where_param = "{"
-    for (k, v) in params.items():
-        where_param += '"{}":"{}",'.format(k, v)
-    return where_param[:-1] + '}'
-
-
-def save_record_to_db(global_info, startup_times):
-    """ Save the record information in the mongo db. The information saved is
-    composed by joining both dicts received as parameter
-    """
-    logger.info('Saving records to db')
-    for instance in startup_times:
-        instance.update(global_info)
-        r = requests.post('{}'.format(BASE_URL),
-                          json=instance,
-                          headers=get_headers())
-        if not r.status_code == requests.codes.created:
-            logger.error('Error uploading instance data: {}'.format(instance))
-            logger.error(r.status_code, r.reason)
-
-
-def is_run_saved(run_id):
-    """ Indicate if there is at least one record with the run_id saved in the
-    mongo db
-    """
-    where_param = _get_where_param(run_id=run_id)
-    r = requests.get(BASE_URL, params={'where': where_param},
-                     headers=get_headers())
-    if r.status_code == requests.codes.ok:
-        return r.json()['_meta']['total'] > 0
-    else:
-        logger.error('Error retrieving information for run: {}'.format(run_id))
-        logger.error(r.status_code, r.reason)
-
-
-def get_records_for_app(app, device, channel):
-    """ Retrieve the records from the mongo db for an specific app, device and
-    channel. The records by default are ordered by build number
-    :param app: The app used to filter the results
-    :param device: The device used to filter the results
-    :param channel: The channel used to filter the results
-    """
-    where_param = _get_where_param(app=app, device=device, channel=channel)
-
-    stop = False
-    page_iter = 1
-    records = []
-    while not stop:
-        r = requests.get(BASE_URL,
-                         params={'where': where_param, 'max_results': 50,
-                                 'page': page_iter},
-                         headers=get_headers())
-        if r.status_code == requests.codes.ok:
-            records.extend(r.json()['_items'])
-        else:
-            logger.error('Error retrieving data from url: {}'.format(r.url))
-            logger.error(r.status_code, r.reason)
-
-        metadata = r.json()['_meta']
-        if metadata['max_results'] * metadata['page'] >= metadata['total']:
-            stop = True
-        else:
-            page_iter += 1
-
-    return records

=== removed file 'qakit/appstartup/data_processor.py'
--- qakit/appstartup/data_processor.py	2016-02-18 14:19:19 +0000
+++ qakit/appstartup/data_processor.py	1970-01-01 00:00:00 +0000
@@ -1,74 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import statistics
-
-from qakit.appstartup import config_dict as config
-
-BUILDS_TO_PLOT = config['APP_STARTUP']['BUILDS_TO_PLOT']
-
-
-def get_times_by_build(app_records):
-    """ Calculate the times for the different builds
-    :return: [builds], [mean cold], [stdev cold], [mean hot], [stdev hot]
-    """
-    build_times = {}
-    for record in app_records:
-        build = record['build_number']
-        if build in build_times:
-            build_times[build].append(record)
-        else:
-            build_times[build] = [record]
-
-    times = _process_build_times(build_times)
-    if times:
-        return zip(*times)
-    else:
-        return (), (), (), (), ()
-
-
-def _process_build_times(build_times):
-    """ Process and filter build times """
-    times = []
-    for build in sorted(build_times)[- int(BUILDS_TO_PLOT):]:
-        hot_times = _get_times_by_type(build_times[build], 'hot')
-        cold_times = _get_times_by_type(build_times[build], 'cold')
-
-        times.append((build, _get_average(cold_times),
-                      _get_stdev(cold_times),
-                      _get_average(hot_times),
-                      _get_stdev(hot_times)))
-    return times
-
-
-def _get_average(times):
-    if times:
-        return statistics.mean(times)
-    else:
-        return 0
-
-
-def _get_stdev(times):
-    if len(times) >= 2:
-        return statistics.stdev(times)
-    else:
-        return 0
-
-
-def _get_times_by_type(app_records, type):
-    return [app_record['time'] for app_record in app_records if
-            app_record['type'] == type]

=== removed directory 'qakit/appstartup/eve'
=== removed file 'qakit/appstartup/eve/__init__.py'
--- qakit/appstartup/eve/__init__.py	2016-01-19 20:21:59 +0000
+++ qakit/appstartup/eve/__init__.py	1970-01-01 00:00:00 +0000
@@ -1,16 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== removed file 'qakit/appstartup/eve/run.py'
--- qakit/appstartup/eve/run.py	2016-01-19 20:21:59 +0000
+++ qakit/appstartup/eve/run.py	1970-01-01 00:00:00 +0000
@@ -1,22 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-from eve import Eve
-app = Eve()
-
-if __name__ == '__main__':
-    app.run()

=== removed file 'qakit/appstartup/eve/settings.py'
--- qakit/appstartup/eve/settings.py	2016-01-28 05:47:05 +0000
+++ qakit/appstartup/eve/settings.py	1970-01-01 00:00:00 +0000
@@ -1,89 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-
-MONGO_HOST = os.getenv('EVE_MONGO_HOST', '127.0.0.1')
-MONGO_PORT = os.getenv('EVE_MONGO_PORT', '27017')
-MONGO_USERNAME = os.getenv('EVE_MONGO_USERNAME', 'ubuntuUser')
-MONGO_PASSWORD = os.getenv('EVE_MONGO_PASSWORD', 'ubuntuPassword')
-MONGO_DBNAME = os.getenv('EVE_MONGO_DBNAME', 'test')
-SERVER_NAME = os.getenv('EVE_SERVER_NAME', None)
-
-
-appstartup_schema = {
-    # Schema definition, based on Cerberus grammar. Check the Cerberus project
-    # (https://github.com/nicolaiarocci/cerberus) for details.
-    'run_id': {
-        'type': 'string',
-        'required': True,
-    },
-    'time': {
-        'type': 'number',
-        'required': True,
-    },
-    'type': {
-        'type': 'string',
-        'required': True,
-    },
-    'iteration': {
-        'type': 'number',
-        'required': True,
-    },
-    'app': {
-        'type': 'string',
-        'required': False,
-    },
-    'build_number': {
-        'type': 'string',
-        'required': True,
-    },
-    'device': {
-        'type': 'string',
-        'required': True,
-    },
-    'channel': {
-        'type': 'string',
-        'required': True,
-    },
-}
-
-appstartup = {
-    # 'title' tag used in item links. Defaults to the resource title minus
-    # the final, plural 's' (works fine in most cases but not for 'people')
-    'item_title': 'appstartup',
-
-    # by default the standard item entry point is defined as
-    # '/people/<ObjectId>'. We leave it untouched, and we also enable an
-    # additional read-only entry point. This way consumers can also perform
-    # GET requests at '/people/<lastname>'.
-    'additional_lookup': {
-        'url': 'regex("[\w]+")',
-        'field': 'app'
-    },
-
-    # We choose to override global cache-control directives for this resource.
-    'cache_control': 'max-age=10,must-revalidate',
-    'cache_expires': 10,
-
-    # most global settings can be overridden at resource level
-    'schema': appstartup_schema,
-}
-
-DOMAIN = {'appstartup': appstartup}
-RESOURCE_METHODS = ['GET', 'POST', 'DELETE']
-ITEM_METHODS = ['GET', 'PATCH', 'PUT', 'DELETE']

=== removed file 'qakit/appstartup/orchestrator.py'
--- qakit/appstartup/orchestrator.py	2016-02-12 17:01:08 +0000
+++ qakit/appstartup/orchestrator.py	1970-01-01 00:00:00 +0000
@@ -1,99 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-from argparse import ArgumentParser
-import logging
-import os
-
-from qakit.appstartup import allowed_apps
-from qakit.appstartup import config_dict as config
-from qakit.appstartup import dao
-from qakit.appstartup import data_processor
-from qakit.appstartup import parser
-from qakit.appstartup import plotter
-from qakit.appstartup import report
-from qakit.practitest import subunit_results
-
-logger = logging.getLogger(__name__)
-
-HTML_REPORT = config['APP_STARTUP']['HTML_REPORT']
-JSON_REPORT = config['APP_STARTUP']['JSON_REPORT']
-
-
-def save_results_if_needed(global_info, startup_times):
-    if startup_times:
-        run_id = global_info['run_id']
-        if not dao.is_run_saved(run_id):
-            dao.save_record_to_db(global_info, startup_times)
-        else:
-            logger.warning(
-                'Test results already in the db for the run_id {}'.format(
-                    run_id))
-
-
-def _collect_records_by_app(global_info):
-    app_records = {}
-    device = global_info['device']
-    channel = global_info['channel']
-    for app in allowed_apps:
-        app_records[app] = dao.get_records_for_app(app, device, channel)
-    return app_records
-
-
-def _plot_records_by_apps(records_by_apps, results_dir):
-    app_plots = {}
-
-    for app in allowed_apps:
-        (builds, cold_times, cold_stdevs, hot_times, hot_stdevs) = \
-            data_processor.get_times_by_build(records_by_apps[app])
-        app_plots[app] = plotter.plot_app_records(app, builds, cold_times,
-                                                  cold_stdevs, hot_times,
-                                                  hot_stdevs, results_dir)
-    return app_plots
-
-
-def _create_graphics_dir(results_dir):
-    dir = os.path.join(results_dir, 'appstartup_data')
-    if not os.path.exists(dir):
-        os.mkdir(dir)
-    return dir
-
-
-def generate_reports(subunit_log, global_info, records_by_apps):
-    if HTML_REPORT or JSON_REPORT:
-        graphics_dir = _create_graphics_dir(os.path.dirname(subunit_log))
-        pics = _plot_records_by_apps(records_by_apps, graphics_dir)
-        if HTML_REPORT:
-            report.create_html_report(pics, os.path.dirname(subunit_log),
-                                      report_info=global_info)
-        if JSON_REPORT:
-            report.create_json_report(pics, os.path.dirname(subunit_log),
-                                      report_info=global_info)
-
-
-def main():
-    args_parser = ArgumentParser("Parse app startup times from logs.")
-    args_parser.add_argument("subunit_log")
-    args = args_parser.parse_args()
-    results_list = subunit_results.parse(args.subunit_log)
-    global_info, startup_times = parser.parse_startup_times(results_list)
-    save_results_if_needed(global_info, startup_times)
-    records_by_apps = _collect_records_by_app(global_info)
-    generate_reports(args.subunit_log, global_info, records_by_apps)
-
-if __name__ == "__main__":
-    main()

=== removed file 'qakit/appstartup/parser.py'
--- qakit/appstartup/parser.py	2016-01-22 20:54:59 +0000
+++ qakit/appstartup/parser.py	1970-01-01 00:00:00 +0000
@@ -1,65 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import ast
-
-
-def parse_startup_times(results):
-    """ Retrieve the global run information and a list with all the history
-    instances parsed from the results_list in the subunit test log
-    """
-    startup_times = []
-    tests_logs = _get_results_test_logs(results)
-    global_info = _get_global_information(tests_logs)
-
-    for test_log in tests_logs:
-        times = _get_times(test_log)
-        if global_info and times:
-            startup_times.extend(times)
-    return global_info, startup_times
-
-
-def _get_global_information(tests_logs):
-    for test_log in tests_logs:
-        markers = [line for line in test_log if '<=' in line]
-        if markers:
-            values = markers[0].split('<=')[1].strip()
-            return ast.literal_eval(values)
-    raise RuntimeError('Global info not provided for tests')
-
-
-def _get_results_test_logs(results):
-    test_logs = []
-    for result in results:
-        test_log = result['details'].get('test-log')
-        if test_log:
-            test_logs.append(test_log.as_text().split('\n'))
-    return test_logs
-
-
-def _get_times(test_log):
-    markers = [line for line in test_log if '=>' in line]
-    times = []
-
-    iteration = 1
-    for mark in markers:
-        values = mark.split('=>')[1].strip()
-        values_dict = ast.literal_eval(values)
-        values_dict['iteration'] = iteration
-        times.append(values_dict)
-        iteration += 1
-    return times

=== removed file 'qakit/appstartup/plotter.py'
--- qakit/appstartup/plotter.py	2016-02-10 20:18:30 +0000
+++ qakit/appstartup/plotter.py	1970-01-01 00:00:00 +0000
@@ -1,63 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import matplotlib.pyplot as plt
-import numpy as np
-import os
-
-
-def plot_app_records(app, builds, cold_times, cold_stdevs, hot_times,
-                     hot_stdevs, plots_dir):
-    """ Generate the graphic and return the graphic path.
-    It is supposed the records are already ordered by build number.
-    """
-
-    # Set descriptions
-    fig, ax = plt.subplots()
-    ax.set_title('Times for {} app'.format(app))
-    ax.set_xlabel('Build Number')
-    ax.set_ylabel('Time (seconds)')
-
-    # Config the graphic
-    ind = np.arange(len(builds))
-    width = 0.35
-    ax.set_xticks(ind + width)
-    ax.set_xticklabels(builds)
-    ax.set_xmargin(0.1)
-    ax.yaxis.grid(True)
-
-    rects_cold = ax.bar(ind, cold_times, width, color='#3366FF',
-                        yerr=cold_stdevs, ecolor='#FF1A00', capsize=5,
-                        error_kw={'elinewidth': 2})
-    rects_hot = ax.bar(ind + width, hot_times, width, color='#FF1A00',
-                       yerr=hot_stdevs, ecolor='#3366FF', capsize=5,
-                       error_kw={'elinewidth': 2})
-
-# Shrink current axis's height by 10% on the bottom
-    box = ax.get_position()
-    ax.set_position([box.x0, box.y0 + box.height * 0.1,
-                     box.width, box.height * 0.9])
-
-    if rects_cold and rects_hot:
-        ax.legend((rects_cold[0], rects_hot[0]), ('Cold Start', 'Hot Start'),
-                  loc='upper center', bbox_to_anchor=(0.85, -0.05),
-                  fancybox=True, shadow=True)
-
-    # Save the graphic
-    dst_file = os.path.join(plots_dir, 'barchart_{}.png'.format(app))
-    plt.savefig(dst_file)
-    return dst_file

=== removed file 'qakit/appstartup/report.py'
--- qakit/appstartup/report.py	2016-02-12 17:08:11 +0000
+++ qakit/appstartup/report.py	1970-01-01 00:00:00 +0000
@@ -1,72 +0,0 @@
-#!/usr/bin/python3
-# UESQA Metrics
-# Copyright (C) 2016 Canonical
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import json
-import os
-
-html_header = '<html><head><title>{}</title></head><body><div align="center"><table>'  # NOQA
-html_footer = '</table></div></body></html>'
-html_picture = '<tr><td><br/><br/><font size="7" color="#8B005A"><center>{}</center></font><br/><img src="{}"/></td></tr>'  # NOQA
-html_logo = '<img src="http://design.ubuntu.com/wp-content/uploads/canonical-logo1.png">'  # NOQA
-html_title = '<br/><br/><div><font size="7" color="#8B005A">Apps StartUp Execution Report</font></div>'  # NOQA
-html_info_title_header = '<br/><br/><div><font size="6" color="#8B005A">General information</font>'  # NOQA
-html_info_title_footer = '</div>'
-html_info = '<div><br/><font size="4">{}: {}</font></div>'
-
-
-def create_html_report(out_files, report_dir,
-                       report_file="app_startup_report.html",
-                       report_info=None):
-    report_path = os.path.join(report_dir, report_file)
-
-    with open(report_path, "w") as report:
-        # write out the html header
-        report.write(html_header.format('App startup times report'))
-        report.write(html_logo)
-
-        report.write(html_title)
-        report.write(html_info_title_header)
-        for info in sorted(report_info.keys()):
-            report.write(html_info.format(info.replace('_', ' ').title(),
-                                          report_info[info]))
-        report.write(html_info_title_footer)
-
-        for app, path in out_files.items():
-            report.write(html_picture.format(app.title(), path))
-
-        report.write(html_footer)
-
-    return report_path
-
-
-def create_json_report(out_files, report_dir,
-                       report_file="app_startup_report.json",
-                       report_info=None):
-
-    report_path = os.path.join(report_dir, report_file)
-    data = dict()
-    data['channel'] = report_info['channel']
-    data['device'] = report_info['device']
-    data['build_number'] = report_info['build_number']
-
-    charts = []
-    for app, path in out_files.items():
-        charts.append({"title": app.title(), "path": path})
-    data['charts'] = charts
-
-    with open(report_path, "w") as report:
-        report.write(json.dumps(data, ensure_ascii=False))

=== modified file 'qakit/dashboard/css/kpi.css'
--- qakit/dashboard/css/kpi.css	2016-02-11 21:23:21 +0000
+++ qakit/dashboard/css/kpi.css	2016-04-22 20:41:08 +0000
@@ -27,6 +27,6 @@
     list-style-type: none !important;
 }
 
-.app-startup-elem {
+.perf-elem {
     text-align: center;
 }

=== modified file 'qakit/dashboard/index.html'
--- qakit/dashboard/index.html	2016-02-12 17:24:39 +0000
+++ qakit/dashboard/index.html	2016-04-22 20:41:08 +0000
@@ -27,6 +27,8 @@
 	<script src="js/test-library.js"></script>
 	<script src="js/test-plans.js"></script>
 	<script src="js/appstartup.js"></script>
+	<script src="js/scalability.js"></script>
+	<script src="js/performance.js"></script>
 
 <style type="text/css">
     .bs-example{
@@ -104,6 +106,20 @@
         <ul id="app-startup-sublinks" class="sub-links">
         </ul>
       </div>
+	  <div class="link-blue" id="scalability-link">
+        <a href="#scalability-section">
+          <i class="fa fa-tachometer"></i>Scalability
+        </a>
+        <ul id="scalability-sublinks" class="sub-links">
+        </ul>
+      </div>
+      <div class="link-blue" id="performance-link">
+        <a href="#performance-section">
+          <i class="fa fa-tachometer"></i>Performance
+        </a>
+        <ul id="performance-sublinks" class="sub-links">
+        </ul>
+      </div>
     </div>
   </aside>
   <div id="main-content" class="main-content">
@@ -279,6 +295,39 @@
       </div>
     </div> <!-- app-startup-section -->
 
+	<div id="scalability-section" class="content-section">
+      <div class="panel panel-default">
+	     <div class="panel-heading"><h1>Scalability Report</h1></div>
+	     <div class="panel-body">
+			 <div class="well well-sm">
+			    <h3 id="scalability-device"></h3>
+			    <h3 id="scalability-channel"></h3>
+			    <h3 id="scalability-build-number"></h3>
+				<p><font color="#FFBF00">Load 1024 elements</font></p>
+				<p><font color="#81F7D8">Load 2048 elements</font></p>
+				<p><font color="#9F81F7">Load 4096 elements</font></p>
+				<p><font color="#F781BE">Load 8192 elements</font></p>
+                <p><font color="#0040FF">Load 16384 elements</font></p>
+			 </div>
+			 <table id="ScalabilityTable" class="table table-striped table-bordered"></table>
+	     </div>
+      </div>
+    </div> <!-- scalability-section -->
+
+ 	<div id="performance-section" class="content-section">
+      <div class="panel panel-default">
+	     <div class="panel-heading"><h1>Performance Report</h1></div>
+	     <div class="panel-body">
+			 <div class="well well-sm">
+			    <h3 id="performance-device"></h3>
+			    <h3 id="performance-channel"></h3>
+			    <h3 id="performance-build-number"></h3>
+			 </div>
+			 <table id="PerformanceTable" class="table table-striped table-bordered"></table>
+	     </div>
+      </div>
+    </div> <!-- performance-section -->
+
     <div id="applications-manual-section" class="content-section">
       <div class="panel panel-default">
 	<div class="panel-heading"><h1>Tests by Application Manual</h1></div>
@@ -326,25 +375,15 @@
     </div> <!-- main-content -->
 
     <script>
-        $(function () {
-            var links = $('.sidebar-links > div');
-            links.on('click', function () {
-                links.removeClass('selected');
-                $(this).addClass('selected');
-            });
-        });
-    </script>
-    <script>
-
-// show only the content-section of id matching hash
-$(window).on("hashchange", function (e) {
-	moveToSection();
-
-});
-
-window.onload = function() {
-    drawSections(true);
-}
+		// show only the content-section of id matching hash
+		window.onhashchange = function (e) {
+			moveToSection();
+		}
+
+		window.onload = function() {
+			selectSection()
+			drawSections(true);
+		}
     </script>
 </body>
 </html>

=== modified file 'qakit/dashboard/js/appstartup.js'
--- qakit/dashboard/js/appstartup.js	2016-02-12 17:24:39 +0000
+++ qakit/dashboard/js/appstartup.js	2016-04-22 20:41:08 +0000
@@ -3,7 +3,7 @@
 function drawAppStartUpSection(firstDraw) {
     $.getJSON("./data/app_startup_report.json", function(data) {
         if (firstDraw){
-            process(data);
+            process_app_startup(data);
         }
     }).fail(function(data, status) {
         console.log("Hiding Appstartup section");
@@ -12,7 +12,7 @@
     });
 }
 
-function process(data) {
+function process_app_startup(data) {
     $('#app-startup-channel').text('Channel: ' + data.channel);
     $('#app-startup-device').text('Device: ' + data.device);
     $('#app-startup-build-number').text('Build Number: ' + data.build_number);
@@ -22,7 +22,7 @@
         title = charts[chart].title
         path = "./data/" + basename(charts[chart].path)
         id = title.toLowerCase() + "-app-startup-section"
-        section = '<div class="app-startup-elem" id="' + id + '" parent-section="app-startup-section"><h2>' + title + '</h2><img src="' + path + '"/></div>'
+        section = '<div class="perf-elem" id="' + id + '" parent-section="app-startup-section"><h2>' + title + '</h2><img src="' + path + '"/></div>'
         $( "#AppStartUpTable" ).append(section);
 
         link = '<li><a href="#' + id + '">' + title + '</a></li>'

=== modified file 'qakit/dashboard/js/main.js'
--- qakit/dashboard/js/main.js	2016-02-11 21:30:40 +0000
+++ qakit/dashboard/js/main.js	2016-04-22 20:41:08 +0000
@@ -9,6 +9,8 @@
     drawApplicationsSection();
     drawDomainsSection();
     drawAppStartUpSection(firstDraw);
+    drawScalabilitySection(firstDraw)
+    drawPerformanceSection(firstDraw)
 
     hideUnusedLinks();
 }
@@ -49,4 +51,12 @@
     if (!window.location.hash){
         drawSections(false);
     }
+}
+
+function selectSection() {
+    var links = $(".sidebar-links > div");
+    links.on('click', function () {
+        links.removeClass('selected');
+        $(this).addClass('selected');
+    });
 }
\ No newline at end of file

=== added file 'qakit/dashboard/js/performance.js'
--- qakit/dashboard/js/performance.js	1970-01-01 00:00:00 +0000
+++ qakit/dashboard/js/performance.js	2016-04-22 20:41:08 +0000
@@ -0,0 +1,31 @@
+
+
+function drawPerformanceSection(firstDraw) {
+    $.getJSON("./data/performance_report.json", function(data) {
+        if (firstDraw){
+            process_performance(data);
+        }
+    }).fail(function(data, status) {
+        console.log("Hiding Performance section");
+        $("#performance-section").hide();
+        $("#performance-link").hide();
+    });
+}
+
+function process_performance(data) {
+    $('#performance-channel').text('Channel: ' + data.channel);
+    $('#performance-device').text('Device: ' + data.device);
+    $('#performance-build-number').text('Build Number: ' + data.build_number);
+
+    charts = data.charts
+    for (var chart in charts) {
+        title = charts[chart].title
+        path = "./data/" + basename(charts[chart].path)
+        id = title.toLowerCase() + "-performance-section"
+        section = '<div class="perf-elem" id="' + id + '" parent-section="performance-section"><h2>' + title + '</h2><img src="' + path + '"/></div>'
+        $( "#PerformanceTable" ).append(section);
+
+        link = '<li><a href="#' + id + '">' + title + '</a></li>'
+        $( "#performance-sublinks" ).append(link);
+    }
+}

=== added file 'qakit/dashboard/js/scalability.js'
--- qakit/dashboard/js/scalability.js	1970-01-01 00:00:00 +0000
+++ qakit/dashboard/js/scalability.js	2016-04-22 20:41:08 +0000
@@ -0,0 +1,31 @@
+
+
+function drawScalabilitySection(firstDraw) {
+    $.getJSON("./data/scalability_report.json", function(data) {
+        if (firstDraw){
+            process_scalability(data);
+        }
+    }).fail(function(data, status) {
+        console.log("Hiding Scalability section");
+        $("#scalability-section").hide();
+        $("#scalability-link").hide();
+    });
+}
+
+function process_scalability(data) {
+    $('#scalability-channel').text('Channel: ' + data.channel);
+    $('#scalability-device').text('Device: ' + data.device);
+    $('#scalability-build-number').text('Build Number: ' + data.build_number);
+
+    charts = data.charts
+    for (var chart in charts) {
+        title = charts[chart].title
+        path = "./data/" + basename(charts[chart].path)
+        id = title.toLowerCase() + "-scalability-section"
+        section = '<div class="perf-elem" id="' + id + '" parent-section="scalability-section"><h2>' + title.replace('_', ' ') + '</h2><img src="' + path + '"/></div>'
+        $( "#ScalabilityTable" ).append(section);
+
+        link = '<li><a href="#' + id + '">' + title.replace('_', ' ') + '</a></li>'
+        $( "#scalability-sublinks" ).append(link);
+    }
+}

=== added directory 'qakit/perf'
=== added file 'qakit/perf/__init__.py'
--- qakit/perf/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/__init__.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,30 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import configparser
+
+from qakit import config as qakit_config
+
+
+def _read_config(config_filepath):
+    """ Parse the config at the given filepath and return it """
+    config_file = configparser.ConfigParser()
+    config_file.read(config_filepath)
+    return config_file
+
+
+config_dict = _read_config(qakit_config.get_config_file_location())

=== added directory 'qakit/perf/engine'
=== added file 'qakit/perf/engine/__init__.py'
--- qakit/perf/engine/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/engine/__init__.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,16 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'qakit/perf/engine/dao.py'
--- qakit/perf/engine/dao.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/engine/dao.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,104 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import logging
+import requests
+from qakit.perf import config_dict as config
+
+BASE_URL = 'http://{}:{}/{}'.format(config['PERF']['EVE_HOST'],
+                                    config['PERF']['EVE_PORT'],
+                                    config['PERF']['EVE_PATH'])
+
+logger = logging.getLogger(__name__)
+
+
+def get_headers():
+    return {'Content-Type': 'application/json'}
+
+
+def _get_where_param(**params):
+    """ Get a where parameter used to filter when getting a query to mongo
+    :param params: A dict with all the parameters used to filter
+    """
+    where_param = "{"
+    for (k, v) in params.items():
+        where_param += '"{}":"{}",'.format(k, v)
+    return where_param[:-1] + '}'
+
+
+def save_record_to_db(global_info, records):
+    """ Save the record information in the mongo db. The information saved is
+    composed by joining both dicts received as parameter
+    """
+    logger.info('Saving records to db')
+    for record in records:
+        record.update(global_info)
+        r = requests.post('{}'.format(BASE_URL), data=json.dumps(record),
+                          headers=get_headers())
+        if not r.status_code == requests.codes.created:
+            logger.error('Error uploading record: {}'.format(record))
+            logger.error(r.status_code, r.reason)
+
+
+def is_run_saved(run_id):
+    """ Indicate if there is at least one record with the run_id saved in the
+    mongo db
+    """
+    where_param = _get_where_param(run_id=run_id)
+    r = requests.get(BASE_URL, params={'where': where_param},
+                     headers=get_headers())
+    if r.status_code == requests.codes.ok:
+        return r.json()['_meta']['total'] > 0
+    else:
+        logger.error('Error retrieving information for run: {}'.format(run_id))
+        logger.error(r.status_code, r.reason)
+
+
+def get_records_for_key(key, value, device, channel):
+    """ Retrieve the records from the mongo db for an specific key and value,
+    device and channel. The records by default are ordered by build number
+    :param device: The device used to filter the results
+    :param channel: The channel used to filter the results
+    :param key: the key used to filter the results
+    :param value: the value used to filter the results
+
+    """
+    args = {'device': device, 'channel': channel, key: value}
+    where_param = _get_where_param(**args)
+
+    stop = False
+    page_iter = 1
+    records = []
+    while not stop:
+        r = requests.get(BASE_URL,
+                         params={'where': where_param, 'max_results': 50,
+                                 'page': page_iter},
+                         headers=get_headers())
+        if r.status_code == requests.codes.ok:
+            records.extend(r.json()['_items'])
+        else:
+            logger.error('Error retrieving data from url: {}'.format(r.url))
+            logger.error(r.status_code, r.reason)
+
+        metadata = r.json()['_meta']
+        if metadata['max_results'] * metadata['page'] >= metadata['total']:
+            stop = True
+        else:
+            page_iter += 1
+
+    return records

=== added file 'qakit/perf/engine/data_processor.py'
--- qakit/perf/engine/data_processor.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/engine/data_processor.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,81 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import statistics
+from qakit.perf import config_dict as config
+
+BUILDS_TO_PLOT = config['PERF']['BUILDS_TO_PLOT']
+
+
+class BuildTime:
+
+    def __init__(self, build, time, stdev):
+        self.build = build
+        self.time = time
+        self.stdev = stdev
+
+
+def get_times_by_build(records, key, values):
+    """ Calculate the times for the different builds and values
+    :param records: the set of time records used to get the desired times
+    :param key: the key to filter
+    :param values: the values to group the results
+    :return: a dict of BuildTime objects by key
+    """
+    build_times = {}
+    for record in records:
+        build = record['build_number']
+        if build in build_times:
+            build_times[build].append(record)
+        else:
+            build_times[build] = [record]
+
+    times = {}
+    for value in values:
+        times[value] = _process_build_times(build_times, key, value)
+
+    return times
+
+
+def _process_build_times(build_times, key, value):
+    """ Process and filter build times """
+    times = []
+    for build in sorted(build_times)[- int(BUILDS_TO_PLOT):]:
+        times_by_key = _get_times_by_key(build_times[build], key, value)
+
+        times.append(BuildTime(build,
+                               _get_average(times_by_key),
+                               _get_stdev(times_by_key)))
+    return times
+
+
+def _get_average(times):
+    if times:
+        return statistics.mean(times)
+    else:
+        return 0
+
+
+def _get_stdev(times):
+    if len(times) >= 2:
+        return statistics.stdev(times)
+    else:
+        return 0
+
+
+def _get_times_by_key(records, key, value):
+    return [record['time'] for record in records if record[key] == value]

=== added file 'qakit/perf/engine/parser.py'
--- qakit/perf/engine/parser.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/engine/parser.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,72 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import ast
+import os
+
+
+def parse_log_times(log_lines, times_class):
+    """ Retrieve the global run information and a list with all the history
+    instances parsed from the results_list in the subunit test log
+    :param log_lines: the lines in the log
+    :param times_class: the class used to filter the times to consider
+    :return: the global information of the run and the times for this run
+    """
+    tests_times = []
+    global_info = _get_global_information(log_lines)
+
+    times = _get_times(log_lines, times_class)
+    if global_info and times:
+        tests_times.extend(times)
+    return global_info, tests_times
+
+
+def parse_log_file(log_file):
+    """ Return a list of lines for the log """
+    if not os.path.exists(log_file):
+        raise RuntimeError('Log file does not exist {}'.format(log_file))
+    with open(log_file, 'r') as file:
+        return file.read().splitlines()
+
+
+def _get_global_information(log_lines):
+    markers = [line for line in log_lines if '<=' in line]
+    if markers:
+        values = markers[0].split('<=')[1].strip()
+        return ast.literal_eval(values)
+    raise RuntimeError('Global info not provided for tests')
+
+
+def _get_results_test_logs(results):
+    test_logs = []
+    for result in results:
+        test_log = result['details'].get('test-log')
+        if test_log:
+            test_logs.append(test_log.as_text().split('\n'))
+    return test_logs
+
+
+def _get_times(test_log, times_class):
+    markers = [line for line in test_log if '=>' in line]
+    times = []
+
+    for mark in markers:
+        values = mark.split('=>')[1].strip()
+        values_dict = ast.literal_eval(values)
+        if 'class' in values_dict and values_dict['class'] == times_class:
+            times.append(values_dict)
+    return times

=== added file 'qakit/perf/engine/plotter.py'
--- qakit/perf/engine/plotter.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/engine/plotter.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,93 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import numpy as np
+import os
+
+# Force matplotlib to not use any Xwindows backend.
+import matplotlib
+matplotlib.use('Agg')
+import matplotlib.pyplot as plt  # NOQA
+
+
+class TimesBar:
+
+    def __init__(self, times, stdevs, title, color, stdevcolor):
+        self.times = times
+        self.stdevs = stdevs
+        self.title = title
+        self.color = color
+        self.stdevcolor = stdevcolor
+
+
+def _get_xspacing(width, bars):
+    if len(bars) <= 1:
+        return width / 2
+    else:
+        return (len(bars) - 1) * width
+
+
+def plot_records(name, title, builds, bars, plots_dir):
+    """ Generate the graphic and return the graphic path.
+    It is supposed the records are already ordered by build number.
+    :param name: The name which identifies the graph
+    :param title: The title for the graph
+    :param builds: The list of build numbers to plot
+    :param bars: A list of TimesBar objects used to plot bars
+    :param plots_dir: The dir where the graph is stored
+    :return: The destination path of the created graph
+    """
+
+    # Set descriptions
+    fig, ax = plt.subplots()
+    ax.set_title(title)
+    ax.set_xlabel('Build Number')
+    ax.set_ylabel('Time (seconds)')
+
+    # Config the graphic
+    ind = np.arange(len(builds))
+    width = 0.35
+
+    ax.set_xticks(ind + _get_xspacing(width, bars))
+    ax.set_xticklabels(builds)
+    ax.set_xmargin(0.1)
+    ax.yaxis.grid(True)
+
+    rects = []
+    for bar in bars:
+        if bar.times:
+            rects.append(ax.bar(ind + bars.index(bar) * width, bar.times,
+                                width, color=bar.color, yerr=bar.stdevs,
+                                ecolor=bar.stdevcolor, capsize=5,
+                                error_kw={'elinewidth': 2}))
+
+    # Shrink current axis's height by 10% on the bottom
+    box = ax.get_position()
+    ax.set_position([box.x0, box.y0 + box.height * 0.1,
+                     box.width, box.height * 0.9])
+
+    if rects and len(rects) <= 2:
+        ax.legend([rect[0] for rect in rects],
+                  [bar.title for bar in bars],
+                  loc='upper center', bbox_to_anchor=(0.85, -0.05),
+                  fancybox=True, shadow=True)
+
+    # Save the graphic
+    dst_file = os.path.join(plots_dir, 'barchart_{}.png'.format(name))
+    plt.savefig(dst_file)
+    return dst_file

=== added file 'qakit/perf/engine/report.py'
--- qakit/perf/engine/report.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/engine/report.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,54 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import os
+
+
+def create_json_report(out_files, report_dir, report_file, report_info):
+    report_path = os.path.join(report_dir, report_file)
+
+    data = dict()
+    data['channel'] = report_info['channel']
+    data['device'] = report_info['device']
+    data['build_number'] = report_info['build_number']
+
+    charts = []
+    if os.path.exists(report_path):
+        charts = get_charts_already_created(report_path, data)
+        os.remove(report_path)
+
+    for name, path in out_files.items():
+        charts.append({"title": name.title().replace('_', ' '), "path": path})
+    data['charts'] = charts
+
+    with open(report_path, "w") as report:
+        report.write(json.dumps(data, ensure_ascii=False))
+
+
+def get_charts_already_created(report_path, current_data):
+    try:
+        with open(report_path) as report_file:
+            report_data = json.load(report_file)
+            if report_data['channel'] == current_data['channel'] and \
+               report_data['device'] == current_data['device'] and \
+               report_data['build_number'] == current_data['build_number']:
+                return report_data['charts']
+            else:
+                return []
+    except Exception:
+        return []

=== added directory 'qakit/perf/eve'
=== added file 'qakit/perf/eve/__init__.py'
--- qakit/perf/eve/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/eve/__init__.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,16 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.

=== added file 'qakit/perf/eve/run.py'
--- qakit/perf/eve/run.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/eve/run.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,24 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from eve import Eve
+
+app = Eve()
+
+
+if __name__ == '__main__':
+    app.run()

=== added file 'qakit/perf/eve/settings.py'
--- qakit/perf/eve/settings.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/eve/settings.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,92 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+
+MONGO_HOST = os.getenv('EVE_MONGO_HOST', '127.0.0.1')
+MONGO_PORT = os.getenv('EVE_MONGO_PORT', '27017')
+MONGO_USERNAME = os.getenv('EVE_MONGO_USERNAME', 'ubuntuUser')
+MONGO_PASSWORD = os.getenv('EVE_MONGO_PASSWORD', 'ubuntuPassword')
+MONGO_DBNAME = os.getenv('EVE_MONGO_DBNAME', 'test')
+SERVER_NAME = os.getenv('EVE_SERVER_NAME', None)
+
+perf_schema = {
+    # Schema definition, based on Cerberus grammar. Check the Cerberus project
+    # (https://github.com/nicolaiarocci/cerberus) for details.
+    'run_id': {
+        'type': 'string',
+        'required': True,
+    },
+    'class': {
+        'type': 'string',
+        'required': True,
+    },
+    'time': {
+        'type': 'number',
+        'required': True,
+    },
+    'type': {
+        'type': 'string',
+        'required': False,
+    },
+    'name': {
+        'type': 'string',
+        'required': True,
+    },
+    'load': {
+        'type': 'number',
+        'required': False,
+    },
+    'build_number': {
+        'type': 'string',
+        'required': True,
+    },
+    'device': {
+        'type': 'string',
+        'required': True,
+    },
+    'channel': {
+        'type': 'string',
+        'required': True,
+    },
+}
+
+perf = {
+    # 'title' tag used in item links. Defaults to the resource title minus
+    # the final, plural 's' (works fine in most cases but not for 'people')
+    'item_title': 'perf',
+
+    # by default the standard item entry point is defined as
+    # '/people/<ObjectId>'. We leave it untouched, and we also enable an
+    # additional read-only entry point. This way consumers can also perform
+    # GET requests at '/people/<lastname>'.
+    'additional_lookup': {
+        'url': 'regex("[\w]+")',
+        'field': 'name'
+    },
+
+    # We choose to override global cache-control directives for this resource.
+    'cache_control': 'max-age=10,must-revalidate',
+    'cache_expires': 10,
+
+    # most global settings can be overridden at resource level
+    'schema': perf_schema,
+}
+
+DOMAIN = {'perf': perf}
+RESOURCE_METHODS = ['GET', 'POST', 'DELETE']
+ITEM_METHODS = ['GET', 'PATCH', 'PUT', 'DELETE']

=== added directory 'qakit/perf/orchestration'
=== added file 'qakit/perf/orchestration/__init__.py'
--- qakit/perf/orchestration/__init__.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/orchestration/__init__.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,84 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from argparse import ArgumentParser
+import logging
+import os
+
+from qakit.practitest import subunit_results
+from qakit.perf import config_dict as config
+from qakit.perf.engine import dao, parser
+
+logger = logging.getLogger(__name__)
+
+json_report = config['PERF']['JSON_REPORT']
+
+
+def save_results_if_needed(global_info, times):
+    if times:
+        run_id = global_info['run_id']
+        if not dao.is_run_saved(run_id):
+            dao.save_record_to_db(global_info, times)
+        else:
+            logger.warning(
+                'Test results already in the db for the run_id {}'.format(
+                    run_id))
+
+
+def collect_records_by_name(global_info, allowed_names):
+    records = {}
+    device = global_info['device']
+    channel = global_info['channel']
+    for name in allowed_names:
+        records[name] = dao.get_records_for_key('name', name, device, channel)
+    return records
+
+
+def _parse_input(input_class, parser_method):
+    args_parser = ArgumentParser("Parse times from logs.")
+    args_parser.add_argument("log")
+    args = args_parser.parse_args()
+
+    log_lines = parser_method(args.log)
+    output_dir = os.path.dirname(args.log)
+    return parser.parse_log_times(log_lines, input_class), output_dir
+
+
+def parse_subunit_input(input_class):
+    return _parse_input(input_class, subunit_results.get_log_lines)
+
+
+def parse_text_input(input_class):
+    return _parse_input(input_class, parser.parse_log_file)
+
+
+def create_graphics_dir(results_dir, dirname):
+    dir = os.path.join(results_dir, dirname)
+    if not os.path.exists(dir):
+        os.mkdir(dir)
+    return dir
+
+
+def get_from_build_times(build_times, key, attr):
+    """ Get the list of values stored in the attr for every BuildTime stored
+    for the desired key in the build_times
+    """
+    return [getattr(time, attr) for time in build_times[key]]
+
+
+def strip_string_list(string_list):
+    return [string.strip() for string in string_list.split(',')]

=== added file 'qakit/perf/orchestration/app_startup.py'
--- qakit/perf/orchestration/app_startup.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/orchestration/app_startup.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,79 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+
+from qakit.perf import config_dict as config
+from qakit.perf.engine import report, plotter, data_processor
+from qakit.perf import orchestration as orch
+
+logger = logging.getLogger(__name__)
+
+eve_path = 'app_startup'
+allowed_apps = orch.strip_string_list(config['APP_STARTUP']['ALLOWED_APPS'])
+types = orch.strip_string_list(config['APP_STARTUP']['TYPES'])
+titles = orch.strip_string_list(config['APP_STARTUP']['TITLES'])
+colors = orch.strip_string_list(config['APP_STARTUP']['COLORS'])
+ecolors = orch.strip_string_list(config['APP_STARTUP']['ECOLORS'])
+
+
+def _plot_records_by_apps(records_by_apps, results_dir):
+    app_plots = {}
+
+    for app in allowed_apps:
+        title = 'Times for {} app'.format(app)
+        build_times = data_processor.get_times_by_build(records_by_apps[app],
+                                                        'type', types)
+        builds = [times.build for times in build_times[types[0]]]
+
+        bars = []
+        iter = 0
+        for type in types:
+            bars.append(plotter.TimesBar(
+                orch.get_from_build_times(build_times, type, 'time'),
+                orch.get_from_build_times(build_times, type, 'stdev'),
+                titles[iter],
+                '#{}'.format(colors[iter]),
+                '#{}'.format(ecolors[iter])))
+            iter += 1
+
+        app_plots[app] = plotter.plot_records(app, title, builds, bars,
+                                              results_dir)
+
+    return app_plots
+
+
+def generate_reports(output_dir, global_info, records_by_apps):
+    if orch.json_report:
+        graphics_dir = orch.create_graphics_dir(output_dir, 'appstartup_data')
+        pics = _plot_records_by_apps(records_by_apps, graphics_dir)
+        report.create_json_report(pics, output_dir,
+                                  report_file='app_startup_report.json',
+                                  report_info=global_info)
+
+
+def main():
+    (global_info, startup_times), output_dir = orch.parse_subunit_input(
+        'app_startup')
+
+    orch.save_results_if_needed(global_info, startup_times)
+    records_by_apps = orch.collect_records_by_name(global_info, allowed_apps)
+    generate_reports(output_dir, global_info, records_by_apps)
+
+
+if __name__ == "__main__":
+    main()

=== added file 'qakit/perf/orchestration/performance.py'
--- qakit/perf/orchestration/performance.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/orchestration/performance.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,80 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+
+from qakit.perf import config_dict as config
+from qakit.perf.engine import report, plotter, data_processor
+from qakit.perf import orchestration as orch
+
+logger = logging.getLogger(__name__)
+
+eve_path = 'performance'
+allowed_scenarios = orch.strip_string_list(
+    config['PERFORMANCE']['PERFORMANCE_ALLOWED_SCENARIOS'])
+types = orch.strip_string_list(config['PERFORMANCE']['PERFORMANCE_TYPES'])
+titles = orch.strip_string_list(config['PERFORMANCE']['PERFORMANCE_TITLES'])
+colors = orch.strip_string_list(config['PERFORMANCE']['PERFORMANCE_COLORS'])
+ecolors = orch.strip_string_list(config['PERFORMANCE']['PERFORMANCE_ECOLORS'])
+
+
+def _plot_records_by_scenarios(records_by_apps, results_dir):
+    plots = {}
+
+    for scenario in allowed_scenarios:
+        title = 'Times for {} scenario'.format(scenario)
+        build_times = data_processor.get_times_by_build(
+            records_by_apps[scenario], 'type', types)
+        builds = [times.build for times in build_times[types[0]]]
+
+        bars = []
+        iter = 0
+        for type in types:
+            bars.append(plotter.TimesBar(
+                orch.get_from_build_times(build_times, type, 'time'),
+                orch.get_from_build_times(build_times, type, 'stdev'),
+                titles[iter],
+                '#{}'.format(colors[iter]),
+                '#{}'.format(ecolors[iter])))
+            iter += 1
+
+        plots[scenario] = plotter.plot_records(scenario, title, builds, bars,
+                                               results_dir)
+
+    return plots
+
+
+def generate_reports(output_dir, global_info, records):
+    if orch.json_report:
+        graphics_dir = orch.create_graphics_dir(output_dir, 'performance_data')
+        pics = _plot_records_by_scenarios(records, graphics_dir)
+        report.create_json_report(pics, output_dir,
+                                  report_file='performance_report.json',
+                                  report_info=global_info)
+
+
+def main():
+    (global_info, startup_times), output_dir = orch.parse_text_input(
+        'performance')
+
+    orch.save_results_if_needed(global_info, startup_times)
+    records = orch.collect_records_by_name(global_info, allowed_scenarios)
+    generate_reports(output_dir, global_info, records)
+
+
+if __name__ == "__main__":
+    main()

=== added file 'qakit/perf/orchestration/scalability.py'
--- qakit/perf/orchestration/scalability.py	1970-01-01 00:00:00 +0000
+++ qakit/perf/orchestration/scalability.py	2016-04-22 20:41:08 +0000
@@ -0,0 +1,78 @@
+#!/usr/bin/python3
+# UESQA Metrics
+# Copyright (C) 2016 Canonical
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import logging
+
+from qakit.perf import config_dict as config
+from qakit.perf.engine import report, plotter, data_processor
+from qakit.perf import orchestration as orch
+
+logger = logging.getLogger(__name__)
+
+allowed_names = orch.strip_string_list(config['SCALABILITY']['ALLOWED_NAMES'])
+loads = [int(load) for load in config['SCALABILITY']['LOADS'].split(',')]
+titles = orch.strip_string_list(config['SCALABILITY']['TITLES'])
+colors = orch.strip_string_list(config['SCALABILITY']['COLORS'])
+ecolors = orch.strip_string_list(config['SCALABILITY']['ECOLORS'])
+
+
+def _plot_records_by_name(records_by_apps, results_dir):
+    plots = {}
+
+    for name in allowed_names:
+        title = 'Times for {} scenario'.format(name.replace('_', ' '))
+        build_times = data_processor.get_times_by_build(records_by_apps[name],
+                                                        'load', loads)
+        builds = [times.build for times in build_times[loads[0]]]
+
+        bars = []
+        iter = 0
+        for load in loads:
+            bars.append(plotter.TimesBar(
+                orch.get_from_build_times(build_times, load, 'time'),
+                orch.get_from_build_times(build_times, load, 'stdev'),
+                titles[iter],
+                '#{}'.format(colors[iter]),
+                '#{}'.format(ecolors[iter])))
+            iter += 1
+
+        plots[name] = plotter.plot_records(name, title, builds, bars,
+                                           results_dir)
+
+    return plots
+
+
+def generate_reports(output_dir, global_info, records_by_apps):
+    if orch.json_report:
+        graphics_dir = orch.create_graphics_dir(output_dir, 'scalability_data')
+        pics = _plot_records_by_name(records_by_apps, graphics_dir)
+        report.create_json_report(pics, output_dir,
+                                  report_file='scalability_report.json',
+                                  report_info=global_info)
+
+
+def main():
+    (global_info, scal_times), output_dir = orch.parse_subunit_input(
+        'scalability')
+
+    orch.save_results_if_needed(global_info, scal_times)
+    records_by_apps = orch.collect_records_by_name(global_info, allowed_names)
+    generate_reports(output_dir, global_info, records_by_apps)
+
+
+if __name__ == "__main__":
+    main()

=== modified file 'qakit/practitest/subunit_results.py'
--- qakit/practitest/subunit_results.py	2015-04-15 19:03:17 +0000
+++ qakit/practitest/subunit_results.py	2016-04-22 20:41:08 +0000
@@ -41,3 +41,18 @@
         if result['id'] == test_name:
             return result
     return None
+
+
+def get_log_lines(filepath):
+    """ Parse the subunit file at the given path and return all the lines in
+    the log details for all the tests
+    """
+    results = parse(filepath)
+    test_logs = []
+    for result in results:
+        test_log = result['details'].get('test-log')
+        if test_log:
+            test_logs.append(test_log.as_text().split('\n'))
+
+    log_lines = [line for sublist in test_logs for line in sublist]
+    return log_lines

=== modified file 'setup.py'
--- setup.py	2015-12-09 11:33:15 +0000
+++ setup.py	2016-04-22 20:41:08 +0000
@@ -39,7 +39,7 @@
             ('subunit-practitest-export = '
              'qakit.practitest.report_subunit_results_to_practitest:main'),
             ('parse-app-startup = '
-             'qakit.appstartup.parse_startup_times:main'),
+             'qakit.perf.parse_startup_times:main'),
         ]
     }
 )