Merge lp:~coreygoldberg/ubuntu-test-cases/touch-trace-export into lp:ubuntu-test-cases/touch

[Corey Goldberg]

This branch adds a new script: `scripts/app-trace-export.py`, used to parse app traces for benchmarking and export to external MongoDB.

usage: app-trace-export.py [-h] app_id path

Summary:
 * parses binary lttng traces
 * gathers relevant tracepoints and timestamps
 * generates a python dictionary with test run information and gathered tracepoints
 * pretty prints the result dict to console/stdout
 * sends the result to a remote MongoDB

(example with traces stored under ~/lttng-traces/)
run from a branch
  $ bzr branch lp:~coreygoldberg/ubuntu-test-cases/touch-trace-export tracing
  $ cd tracing
  $ ./scripts/app-trace-export.py sudoku-app ~/lttng-traces/sudoku-start-20131217-095828/ust/uid/32011/32-bit/

[lp:~coreygoldberg/ubuntu-test-cases/touch-trace-export]

142. By Corey Goldberg on 2014-02-06

merged trunk

143. By Corey Goldberg on 2014-02-06

refactor and fix pep8

Unmerged revisions

143. By Corey Goldberg on 2014-02-06

refactor and fix pep8

142. By Corey Goldberg on 2014-02-06

merged trunk

141. By Corey Goldberg on 2014-02-04

added yet more fields to json doc

140. By Corey Goldberg on 2014-02-04

added more fields to json doc

139. By Corey Goldberg on 2014-02-03

create json structured doc

138. By Corey Goldberg on 2014-02-03

fixed import

137. By Corey Goldberg on 2014-02-02

send results to mongodb

136. By Corey Goldberg on 2014-02-02

refactor script

135. By Corey Goldberg on 2014-02-02

import teds initial script

134. By Alex Chiang on 2014-01-17

Update tests/customizations/setup.sh to handle updated lp:savilerow project

lp:savilerow now includes the formerly separate tests/ branch, co-locating
both code and tests in the same branch.

Update the script here to bzr export the tests from the new location.

=== added file 'scripts/app-trace-export.py'
--- scripts/app-trace-export.py	1970-01-01 00:00:00 +0000
+++ scripts/app-trace-export.py	2014-02-06 20:39:30 +0000
@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+
+"""
+Export application trace events for app startup.
+"""
+
+import argparse
+import os
+
+from datetime import datetime
+from pprint import pprint
+
+from babeltrace import TraceCollection
+# Install Python bindings for libbabeltrace:
+#     $ sudo add-apt-repository ppa:ted/babeltrace-python
+#     $ sudo apt-get update
+#     $ sudo apt-get install babeltrace python3-libbabeltrace
+
+from pymongo import MongoClient
+# Install Python MongoDB client:
+#     $ sudo apt-get install python3-pymongo
+
+
+def parse_traces(trace_path):
+    """
+    Read binary traces and convert to dict of
+    tracepoint names and timestamps.  Timestamp format is:
+    epoch secs + nanosecs.
+    """
+
+    # Tracepoints to include for app startup
+    tracepoints = (
+        'upstart_app_launch:libual_start_message_sent',
+        'upstart_app_launch:libual_start',
+        'upstart_app_launch:exec_pre_exec',
+        'upstart_app_launch:libual_start_message_callback',
+    )
+
+    # Build up the traces collection
+    traces = TraceCollection()
+    ret = traces.add_trace(trace_path, 'ctf')
+    if ret is None:
+        raise IOError('Error adding trace')
+
+    # Record the timestamps
+    timestamps = {}
+    for event in traces.events:
+        # We only want the event if it's the first occurance to
+        # handle possibly messy traces.
+        if event.name not in timestamps:
+            # Keep only the timestamps we care about by event name
+            if event.name in tracepoints:
+                timestamps[event.name] = event.timestamp
+
+    return timestamps
+
+
+def generate_doc(tracepoints):
+    return dict(
+        app_id=args.app_id,
+        android_serial=os.environ.get('ANDROID_SERIAL'),
+        image_type=os.environ.get('IMAGE_TYPE'),
+        build_num=os.environ.get('BUILD_NUMBER'),
+        build_id=os.environ.get('BUILD_ID'),
+        job_name=os.environ.get('JOB_NAME'),
+        node_name=os.environ.get('NODE_NAME'),
+        datetime=datetime.utcnow(),
+        tracepoints=tracepoints,
+    )
+
+
+def send_to_db(host, port, doc):
+    db = MongoClient(host, port)
+    posts = db.posts
+    post_id = posts.insert(doc)
+    return post_id
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('app_id', help='application identifier')
+    parser.add_argument('path', help='path to trace')
+    args = parser.parse_args()
+
+    tracepoints = parse_traces(args.path),
+    doc = generate_doc(tracepoints)
+
+    pprint(doc)
+
+    send_to_db('localhost', 27017, doc)