Merge lp:~glcompbench-dev/glcompbench/manmower-options-3 into lp:glcompbench

[Marc Ordinas i Llopis]

* Allow tracking both "total" and "subset" collections of profiling data.
* Print out full run statistics at the end of execution.
* Add an option to enable vsync and enable it for composite-canvas-egl.
* Add an extended stats command line option to print min and max profiling information.
* Allow specifying size and position with an X geometry string.
* Add pass/fail testing.
* Add standard deviation.

Also reverted option parsing changes from previous submission.

[lp:~glcompbench-dev/glcompbench/manmower-options-3]

90. By Marc Ordinas i Llopis on 2012-05-01

Options: Call eglSwapInterval only if vsync wasn't enabled.

91. By Marc Ordinas i Llopis on 2012-05-01

Style: Move standard deviation calculation to extended_stats code path.

92. By Marc Ordinas i Llopis on 2012-05-01

Style: Use Util::split to separate points instead of strtok.

[Marc Ordinas i Llopis (marcoil)]

Based on suggestions by Jesse, I've pushed some changes:
* Call eglSwapInterval only if --enable-vsync wasn't requestes.
* Moved standard deviation variable to the scope it's used in.
* Use Util::split in libmatrix instead of strtok.

[Alexandros Frantzis (afrantzis)]

I am still confused about why we need to have multiple substats. In the current code base the SUBSET group ends up holding exactly the same data as the TOTAL group (i.e. Profile::sample_point() samples both substat groups and it's the only public method to sample points).

In case such functionality is indeed needed, I believe that it could be achieved more cleanly by using multiple Profiler instances to hold different data sets.

[Derek Foreman (derek-foreman)]

> I am still confused about why we need to have multiple substats. In the
> current code base the SUBSET group ends up holding exactly the same data as
> the TOTAL group (i.e. Profile::sample_point() samples both substat groups and
> it's the only public method to sample points).

The subsets are temporal - the TOTAL group is quite different from SUBSET after the first iteration. The SUBSET group is what was previously tracked.

The TOTAL group is what I use for pass/fail tests in later commits in the series.

Both contain results accumulated from the same profiler points, but TOTAL is accumulated over the total duration of a test.

> In case such functionality is indeed needed, I believe that it could be
> achieved more cleanly by using multiple Profiler instances to hold different
> data sets.

I think that would result in more convoluted code everywhere that actually uses a profiler point - doubling up every profiler point occurrence via cut and paste, and making some of the resets conditional.

If that's what you'd prefer, I can write it that way instead... Please let me know

[Alexandros Frantzis (afrantzis)]

Thanks for the explanation, it makes much more sense now. You are right that for this use case using separate Profiler instances would be messy.

Based on your patch I have implemented what is essentially a more generic version of your proposal:

https://code.launchpad.net/~linaro-graphics-wg/glcompbench/profiler-stat-collection

The current code in the branch is not complete yet. It only tracks per-iteration statistics, but adding per-test and total statistics is easy. The code is also missing the 'squares' stats datum, but this is trivial to add, too.

In your proposal the total stats are displayed after each test, but they are not really per-test stats, which I find somewhat confusing. I think it would be better to show per-test stats after each test is completed and total stats only once at the end.

Also, I would rather keep the accumulation logic outside of the profiler class. It's easy to calculate accumulated data outside the profiler, given the statistical data in the 'total' collection (calculating std. dev. needs a bit more care).

What do you think?

Meanwhile I will merge into trunk some of the other changes that don't depend on the substats infrastructure.

[Derek Foreman (derek-foreman)]

> Based on your patch I have implemented what is essentially a more generic
> version of your proposal:
>
> https://code.launchpad.net/~linaro-graphics-wg/glcompbench/profiler-stat-
> collection

Very cool :)

> The current code in the branch is not complete yet. It only tracks per-
> iteration statistics, but adding per-test and total statistics is easy. The
> code is also missing the 'squares' stats datum, but this is trivial to add,
> too.
>
> In your proposal the total stats are displayed after each test, but they are
> not really per-test stats, which I find somewhat confusing. I think it would
> be better to show per-test stats after each test is completed and total stats
> only once at the end.

Yes, that sounds quite sensible.

> Also, I would rather keep the accumulation logic outside of the profiler
> class. It's easy to calculate accumulated data outside the profiler, given the
> statistical data in the 'total' collection (calculating std. dev. needs a bit
> more care).
>
> What do you think?

It's not immediately obvious to me how I'd calculate std. dev. anymore. I could easily calculate the std. dev. of individual profiler points, but since there aren't the same number of each profiler point in an iteration, I can't figure out how to calculate std. dev. of test iterations, which is the more useful stat from my perspective.

Though perhaps there's a way that I'm just not seeing?

> Meanwhile I will merge into trunk some of the other changes that don't depend
> on the substats infrastructure.

Thanks!

Unmerged revisions

92. By Marc Ordinas i Llopis on 2012-05-01

Style: Use Util::split to separate points instead of strtok.

91. By Marc Ordinas i Llopis on 2012-05-01

Style: Move standard deviation calculation to extended_stats code path.

90. By Marc Ordinas i Llopis on 2012-05-01

Options: Call eglSwapInterval only if vsync wasn't enabled.

89. By Derek Foreman on 2012-04-27

add standard deviation

88. By Derek Foreman on 2012-04-27

Add pass/fail testing

87. By Derek Foreman on 2012-04-27

Allow specifying size and position with an X geometry string

86. By Derek Foreman on 2012-04-27

Add an extended stats command line option to print min and max profiling information

85. By Derek Foreman on 2012-04-27

Add an option to enable vsync and enable it for composite-canvas-egl

84. By Derek Foreman on 2012-04-27

Print out full run statistics at the end of execution

83. By Derek Foreman on 2012-04-27

Allow tracking both "total" and "subset" collections of profiling data

=== modified file 'src/composite-canvas-egl.cc'
--- src/composite-canvas-egl.cc	2012-04-17 23:19:42 +0000
+++ src/composite-canvas-egl.cc	2012-05-02 14:54:19 +0000
@@ -247,8 +247,9 @@
         return false;
     }
 
-    if (!eglSwapInterval(egl_display_, 0))
-        Log::info("** Failed to set swap interval. Results may be bounded above by refresh rate.\n");
+    if (!Options::vsync)
+        if (!eglSwapInterval(egl_display_, 0))
+            Log::info("** Failed to set swap interval. Results may be bounded above by refresh rate.\n");
 
     use_accel_tfp_ = Options::use_accel_tfp;
 

=== modified file 'src/composite-canvas.cc'
--- src/composite-canvas.cc	2012-04-19 14:15:59 +0000
+++ src/composite-canvas.cc	2012-05-02 14:54:19 +0000
@@ -215,7 +215,7 @@
     attr.event_mask = StructureNotifyMask | ExposureMask;
     mask = CWBackPixel | CWBorderPixel | CWColormap | CWEventMask;
 
-    win = XCreateWindow(xdpy_, root_, 0, 0, width, height,
+    win = XCreateWindow(xdpy_, root_, x, y, width, height,
                         0, vis_info->depth, InputOutput,
                         vis_info->visual, mask, &attr);
 
@@ -226,7 +226,9 @@
         sizehints.y = y;
         sizehints.width  = width;
         sizehints.height = height;
-        sizehints.flags = USSize | USPosition;
+        sizehints.flags = USSize;
+        if (Options::specify_position)
+            sizehints.flags |= USPosition;
         XSetNormalHints(xdpy_, win, &sizehints);
         XSetStandardProperties(xdpy_, win, name, name,
                                None, NULL, 0, &sizehints);
@@ -474,6 +476,8 @@
     CompositeCanvas::profiler_screen_update_pair();
 
     Profiler::instance().reset();
+    if (Options::test_arg)
+        Profiler::instance().init_test(Options::test_arg);
     CompositeCanvas::profiler_cycle_pair().sample_start();
 }
 
@@ -487,6 +491,8 @@
     Profiler &profiler = Profiler::instance();
 
     CompositeCanvas::profiler_cycle_pair().sample_end();
+    profiler.accumulator_end();
+    profiler.accumulator_start();
     CompositeCanvas::profiler_cycle_pair().sample_start();
 
     CompositeCanvas::profiler_cycle_pair().get_stats(stats);
@@ -499,9 +505,18 @@
 
         for (Profiler::Point p = 0; p < profiler.get_num_points(); p += 2) {
             profiler.get_stats(p, p + 1, stats);
-            Log::info("Operation %s => Avg: %f Total: %f Samples: %llu\n",
-                      profiler.get_name_by_point(p),
-                      stats.average, stats.total, stats.nsamples);
+            if (Options::extended_stats) {
+                double sd = 0.0;
+                if (stats.nsamples)
+                    sd = sqrt(stats.squares/(stats.nsamples-1));
+                Log::info("Operation %s => Min: %f Avg: %f Max: %f Sd: %f Total: %f Samples: %llu\n",
+                          profiler.get_name_by_point(p),
+                          stats.low, stats.average, stats.high, sd, stats.total,
+                          stats.nsamples);
+            } else
+                Log::info("Operation %s => Avg: %f Total: %f Samples: %llu\n",
+                          profiler.get_name_by_point(p),
+                          stats.average, stats.total, stats.nsamples);
         }
 
         profiler.reset();
@@ -511,6 +526,24 @@
         }
         iterations_ = 0;
         iteration_complete = true;
+
+        Log::info("Final statistics:\n");
+
+        for (Profiler::Point p = 0; p < profiler.get_num_points(); p += 2) {
+            profiler.get_stats(p, p + 1, stats, Profiler::TOTAL);
+            if (Options::extended_stats) {
+                double sd = 0;
+                if (stats.nsamples)
+                    sd = sqrt(stats.squares/(stats.nsamples-1));
+                Log::info("Operation %s => Min: %f Avg: %f Max: %f Sd: %f Total: %f Samples: %llu\n",
+                          profiler.get_name_by_point(p),
+                          stats.low, stats.average, stats.high, sd, stats.total,
+                          stats.nsamples);
+            } else
+                Log::info("Operation %s => Avg: %f Total: %f Samples: %llu\n",
+                          profiler.get_name_by_point(p),
+                          stats.average, stats.total, stats.nsamples);
+        }
     }
 }
 
@@ -566,9 +599,10 @@
         return false;
 
     /* Create the X window we are going to draw on */
-    width_ = Options::size;
-    height_ = Options::size;
-    canvas_ = create_canvas_x_window(glcompbench_version_string.c_str(), 0, 0,
+    width_ = Options::w;
+    height_ = Options::h;
+    canvas_ = create_canvas_x_window(glcompbench_version_string.c_str(),
+                                     Options::x, Options::y,
                                      width_, height_,
                                      vis_info);
     XFree(vis_info);

=== modified file 'src/glcompbench.cc'
--- src/glcompbench.cc	2012-03-20 16:32:54 +0000
+++ src/glcompbench.cc	2012-05-02 14:54:19 +0000
@@ -41,6 +41,7 @@
 #include "options.h"
 #include "log.h"
 #include "util.h"
+#include "profiler.h"
 
 static const char *default_benchmarks[] = {
     "default",
@@ -276,5 +277,43 @@
 #ifdef USE_GLPROXY
     glProxyClose(proxy_context);
 #endif
+
+    if (Options::test) {
+        double stat = 0.0;
+        double sd = 0.0;
+        Profiler &profiler = Profiler::instance();
+        Profiler::Stats stats;
+        profiler.get_accumulated_stats(stats);
+        if (stats.nsamples == 0) {
+            Log::info("No samples in test subset: TEST FAILED\n");
+            return 1;
+        }
+        sd = sqrt(stats.squares/(stats.nsamples-1));
+        Log::info("Test subset: Min: %f Avg: %f Max: %f Sd: %f Total: %f Samples: %llu\n",
+                  stats.low, stats.average, stats.high, sd, stats.total, stats.nsamples);
+        switch (Options::test_stat) {
+            case Options::STAT_AVERAGE:
+                stat = stats.average;
+                break;
+            case Options::STAT_MINIMUM:
+                stat = stats.low;
+                break;
+            case Options::STAT_MAXIMUM:
+                stat = stats.high;
+                break;
+            case Options::STAT_TOTAL:
+                stat = stats.total;
+                break;
+            case Options::STAT_NONE:
+                break;
+        }
+        if (stat > Options::test_limit) {
+            Log::info("Test statistic value %f > tolerance %f : TEST FAILED\n",
+                      stat, Options::test_limit);
+            return 1;
+        } else
+            Log::info("Test statistic value %f <= tolerance %f : TEST PASSED\n",
+                      stat, Options::test_limit);
+    }
     return 0;
 }

=== modified file 'src/options.cc'
--- src/options.cc	2012-02-17 11:51:23 +0000
+++ src/options.cc	2012-05-02 14:54:19 +0000
@@ -23,6 +23,7 @@
  */
 
 #include <X11/Xlib.h>
+#include <X11/Xutil.h>
 #include <cstring>
 #include <cstdlib>
 #include <cstdio>
@@ -30,7 +31,6 @@
 
 #include "options.h"
 
-int Options::size = 512;
 std::list<Window> Options::window_ids;
 bool Options::run_forever = false;
 bool Options::use_accel_tfp = true;
@@ -43,6 +43,18 @@
 bool Options::show_help = false;
 bool Options::list_tests = false;
 bool Options::swap_buffers = true;
+bool Options::vsync = false;
+bool Options::extended_stats = false;
+int Options::x = 0;
+int Options::y = 0;
+unsigned int Options::w = 512;
+unsigned int Options::h = 512;
+bool Options::specify_position = false;
+bool Options::test = false;
+double Options::test_limit = 0.0;
+char *Options::test_arg = NULL;
+Options::Statistics Options::test_stat = STAT_NONE;
+
 #ifdef USE_GLPROXY
 Options::Backends Options::backend = BACKEND_EGL_ES2;
 #elif defined(USE_GLX)
@@ -68,6 +80,12 @@
     {"debug", 0, 0, 0},
     {"help", 0, 0, 0},
     {"backend", 1, 0, 0},
+    {"enable-vsync", no_argument, 0, 0},
+    {"extended-stats", no_argument, 0, 0},
+    {"geometry", required_argument, 0, 0},
+    {"test-points", required_argument, 0, 0},
+    {"test-limit", required_argument, 0, 0},
+    {"test-stat", required_argument, 0, 0},
     {0, 0, 0, 0}
 };
 
@@ -113,6 +131,20 @@
     return Options::BACKEND_NONE;
 }
 
+static Options::Statistics
+parse_statistic(const char *str)
+{
+    if (!strcmp("average", str))
+        return Options::STAT_AVERAGE;
+    else if (!strcmp("minimum", str))
+        return Options::STAT_MINIMUM;
+    else if (!strcmp("maximum", str))
+        return Options::STAT_MAXIMUM;
+    else if (!strcmp("total", str))
+        return Options::STAT_TOTAL;
+    return Options::STAT_NONE;
+}
+
 void
 Options::print_help()
 {
@@ -136,6 +168,13 @@
            "                         back buffer, use glFinish() instead\n"
            "      --backend BACKEND  The OpenGL backend to use (%s)\n"
            "  -d, --debug            Display debug messages\n"
+           "      --enable-vsync     Enable vertical refresh synchronization\n"
+           "      --extended-stats   Output verbose profiling statistics\n"
+           "      --geometry GEOM    Place window using an X geometry string\n"
+           "Pass/Fail exit status tests:\n"
+           "      --test-points      A comma separated list of profiler points to accumulate\n"
+           "      --test-stat        Statistic for testing (average, minimum, maximum, total)\n"
+           "      --test-limit       upper bound for test statistic\n"
            "  -h, --help             Display help\n",
            available_backends);
 }
@@ -143,6 +182,7 @@
 bool
 Options::parse_args(int argc, char **argv)
 {
+    int bits;
     while (1) {
         int option_index = -1;
         int c;
@@ -160,9 +200,10 @@
 
         if (c == 'i' || !strcmp(optname, "ids"))
             parse_window_ids(optarg, Options::window_ids);
-        else if (c == 's' || !strcmp(optname, "size"))
-            Options::size = strtol(optarg, NULL, 10);
-        else if (!strcmp(optname, "no-accel-tfp"))
+        else if (c == 's' || !strcmp(optname, "size")) {
+            Options::w = strtol(optarg, NULL, 10);
+            Options::h = Options::w;
+        } else if (!strcmp(optname, "no-accel-tfp"))
             Options::use_accel_tfp = false;
         else if (c == 'b' || !strcmp(optname, "benchmark"))
             Options::benchmarks.push_back(optarg);
@@ -186,6 +227,24 @@
             Options::show_help = true;
         else if (!strcmp(optname, "backend"))
             Options::backend = parse_backend(optarg);
+        else if (!strcmp(optname, "enable-vsymc"))
+            Options::vsync = true;
+        else if (!strcmp(optname, "extended-stats"))
+            extended_stats = true;
+        else if (!strcmp(optname, "geometry")) {
+            bits = XParseGeometry(optarg, &x, &y, &w, &h);
+            if ((bits & XValue) || (bits & YValue))
+                specify_position = true;
+        } else if (!strcmp(optname, "test-points")) {
+            Options::test_arg = optarg;
+            Options::test = true;
+	} else if (!strcmp(optname, "test-limit")) {
+            Options::test_limit = strtod(optarg, NULL);
+            Options::test = true;
+        } else if (!strcmp(optname, "test-stat")) {
+            Options::test_stat = parse_statistic(optarg);
+            Options::test = true;
+        }
     }
 
     return true;

=== modified file 'src/options.h'
--- src/options.h	2011-12-12 13:16:47 +0000
+++ src/options.h	2012-05-02 14:54:19 +0000
@@ -34,7 +34,6 @@
     static void print_help();
 
     static std::list<Window> window_ids;
-    static int size;
     static bool run_forever;
     static bool use_accel_tfp;
     static std::list<std::string> benchmarks;
@@ -46,6 +45,16 @@
     static bool show_help;
     static bool list_tests;
     static bool swap_buffers;
+    static bool vsync;
+    static bool extended_stats;
+    static int x;
+    static int y;
+    static unsigned int w;
+    static unsigned int h;
+    static bool specify_position;
+    static bool test;
+    static double test_limit;
+    static char *test_arg;
 
     typedef enum {
         BACKEND_NONE,
@@ -54,6 +63,15 @@
         BACKEND_EGL_ES2
     } Backends;
     static Backends backend;
+
+    typedef enum {
+        STAT_NONE,
+        STAT_AVERAGE,
+        STAT_MINIMUM,
+        STAT_MAXIMUM,
+        STAT_TOTAL
+    } Statistics;
+    static Statistics test_stat;
 };
 
 #endif /* OPTIONS_H_ */

=== modified file 'src/profiler.cc'
--- src/profiler.cc	2011-10-14 12:41:30 +0000
+++ src/profiler.cc	2012-05-02 14:54:19 +0000
@@ -22,40 +22,58 @@
  */
 
 #define __STDC_LIMIT_MACROS
+#include <string.h>
 #include <sys/time.h>
 #include <stdint.h>
 #include <string>
 #include <vector>
 
 #include "profiler.h"
+#include "util.h"
 
 struct ProfilerPoint
 {
+    struct substats {
+        uint64_t last_sample;
+        uint64_t nsamples;
+        uint64_t low;
+        uint64_t high;
+        uint64_t total;
+        double average;
+        double squares;
+    } substats[Profiler::NUM_DURATIONS];
+
     ProfilerPoint(const char *name_):
         name(name_), prev(0), next(0)
     {
-        reset();
+        full_reset();
+    }
+
+    void reset(Profiler::duration d)
+    {
+        substats[d].last_sample = 0;
+        substats[d].nsamples = 0;
+        substats[d].low = UINT64_MAX;
+        substats[d].high = 0;
+        substats[d].total = 0;
+        substats[d].average = 0.0;
+        substats[d].squares = 0.0;
     }
 
     void reset()
     {
-        last_sample = 0;
-        nsamples = 0;
-        low = UINT64_MAX;
-        high = 0;
-        total = 0;
-        average = 0.0;
+        reset(Profiler::SUBSET);
+    }
+
+    void full_reset()
+    {
+        reset(Profiler::SUBSET);
+        reset(Profiler::TOTAL);
     }
 
     std::string name;
     ProfilerPoint *prev;
     ProfilerPoint *next;
-    uint64_t last_sample;
-    uint64_t nsamples;
-    uint64_t low;
-    uint64_t high;
-    uint64_t total;
-    double average;
 };
 
 Profiler::~Profiler()
@@ -81,11 +99,18 @@
 Profiler::Point
 Profiler::register_point(const char *name)
 {
+    Profiler::Point out;
     ProfilerPoint *point = new ProfilerPoint(name);
 
     points_.push_back(point);
-
-    return points_.size() - 1;
+    out = points_.size() - 1;
+    accumulated_stats.nsamples = 0;
+    accumulated_stats.low = UINT64_MAX;
+    accumulated_stats.high = 0;
+    accumulated_stats.total = 0;
+    accumulated_stats.average = 0;
+    accumulated_stats.squares = 0;
+    return out;
 }
 
 /**
@@ -153,13 +178,13 @@
  * @return the found point or NULL
  */
 static ProfilerPoint *
-get_prev_sampled_point(ProfilerPoint *point)
+get_prev_sampled_point(ProfilerPoint *point, Profiler::duration d)
 {
     ProfilerPoint *cur = point->prev;
-    while (cur != NULL && cur->last_sample == 0 && cur != point)
+    while (cur != NULL && cur->substats[d].last_sample == 0 && cur != point)
         cur = cur->prev;
 
-    if (cur != NULL && cur->last_sample != 0)
+    if (cur != NULL && cur->substats[d].last_sample != 0)
         return cur;
     else
         return NULL;
@@ -175,28 +200,45 @@
 bool
 Profiler::sample_point(Profiler::Point p)
 {
+    uint64_t now = get_timestamp_us();
+
     if (!is_valid_point(p))
         return false;
 
+    sample_point(p, TOTAL, now);
+    sample_point(p, SUBSET, now);
+
+    return true;
+}
+
+void
+Profiler::sample_point(Profiler::Point p, Profiler::duration d, uint64_t now)
+{
     ProfilerPoint *point = points_[p];
-    ProfilerPoint *prev = get_prev_sampled_point(point);
-    uint64_t now = get_timestamp_us();
+    ProfilerPoint *prev = get_prev_sampled_point(point, d);
 
     if (prev != NULL) {
-        uint64_t diff = now - point->prev->last_sample;
-        point->average = (point->nsamples * point->average + diff) /
-                         (point->nsamples + 1);
-        point->total += diff;
-        if (diff < point->low)
-            point->low = diff;
-        if (diff > point->high)
-            point->high = diff;
-        point->nsamples++;
+        double oldmean = point->substats[d].average;
+        uint64_t diff = now - point->prev->substats[d].last_sample;
+        if ((d == TOTAL) && (1 << p) & test_mask) {
+            accumulator_active = true;
+            accumulator += diff;
+        }
+        point->substats[d].average =
+                                   (point->substats[d].nsamples *
+                                    point->substats[d].average + diff) /
+                                   (point->substats[d].nsamples + 1);
+        point->substats[d].squares += (diff/1000.0 - oldmean/1000.0) *
+                                     (diff/1000.0 - point->substats[d].average/1000.0);
+        point->substats[d].total += diff;
+        if (diff < point->substats[d].low)
+            point->substats[d].low = diff;
+        if (diff > point->substats[d].high)
+            point->substats[d].high = diff;
+        point->substats[d].nsamples++;
     }
 
-    point->last_sample = now;
-
-    return true;
+    point->substats[d].last_sample = now;
 }
 
 /**
@@ -205,13 +247,15 @@
  * @param p1 the first point
  * @param p2 the second point (may be the same as the first for a complete loop)
  * @param stats the struct to fill
+ * @param duration optionally SUBSET or TOTAL to choose a stats group
  *
  * @return whether the operation succeeded
  */
 bool
 Profiler::get_stats(Profiler::Point p1,
                     Profiler::Point p2,
-                    Stats &stats)
+                    Stats &stats,
+                    enum duration d)
 {
     if (!is_valid_point(p1) || !is_valid_point(p2))
         return false;
@@ -225,18 +269,21 @@
     stats.high = 0.0;
     stats.total = 0.0;
     stats.average = 0.0;
+    stats.squares = 0.0;
 
     /* Go from p1 to p2, gathering stats */
     while (cur != NULL) {
         /*
          * We store values as microseconds internally, but provide them as
-         * milliseconds
+         * milliseconds, except for squares which is already in milliseconds
          */
-        stats.average += cur->average / 1000.0;
-        stats.total += cur->total / 1000.0;
-        stats.low += cur->low / 1000.0;
-        stats.high += cur->high / 1000.0;
-        stats.nsamples = std::max(stats.nsamples, cur->nsamples);
+        stats.average += cur->substats[d].average / 1000.0;
+        stats.squares += cur->substats[d].squares;
+        stats.total += cur->substats[d].total / 1000.0;
+        stats.low += cur->substats[d].low / 1000.0;
+        stats.high += cur->substats[d].high / 1000.0;
+        stats.nsamples = std::max(stats.nsamples,
+                                  cur->substats[d].nsamples);
         if (cur == end)
             break;
         cur = cur->next;
@@ -245,6 +292,25 @@
     return (cur == end);
 }
 
+bool
+Profiler::get_stats(Profiler::Point p1,
+                    Profiler::Point p2,
+                    Stats &stats)
+{
+    return get_stats(p1, p2, stats, SUBSET);
+}
+
+void
+Profiler::get_accumulated_stats(Stats &stats)
+{
+    stats.average = accumulated_stats.average / 1000.0;
+    stats.squares = accumulated_stats.squares;
+    stats.total = accumulated_stats.total / 1000.0;
+    stats.low = accumulated_stats.low / 1000.0;
+    stats.high = accumulated_stats.high / 1000.0;
+    stats.nsamples = accumulated_stats.nsamples;
+}
+
 /**
  * Resets the benchmarking info of a profiling point.
  *
@@ -297,3 +363,51 @@
     return now;
 }
 
+void
+Profiler::accumulator_start()
+{
+    accumulator_active = false;
+    accumulator = 0;
+}
+
+void
+Profiler::accumulator_end()
+{
+    if (!accumulator_active)
+        return;
+
+    accumulator_active = false;
+    double oldmean = accumulated_stats.average;
+    accumulated_stats.average =
+                               (accumulated_stats.nsamples *
+                                accumulated_stats.average + accumulator) /
+                               (accumulated_stats.nsamples + 1);
+    accumulated_stats.squares = accumulated_stats.squares +
+                               (accumulator/1000.0 - oldmean/1000.0) *
+                               (accumulator/1000.0 - accumulated_stats.average/1000.0);
+
+    accumulated_stats.total += accumulator;
+    if (accumulator < accumulated_stats.low)
+            accumulated_stats.low = accumulator;
+    if (accumulator > accumulated_stats.high)
+            accumulated_stats.high = accumulator;
+    accumulated_stats.nsamples++;
+    accumulator = 0;
+}
+
+void
+Profiler::init_test(char *str)
+{
+    test_mask = 0;
+    vector<string> points;
+
+    Util::split(str, ',', points);
+    for (vector<string>::iterator it = points.begin(); it != points.end(); ++it) {
+        /* the +1 is because the points in the profiler pairs both have the same
+         * name, and we want the second point, which actually has the stats...
+         */
+        Profiler::Point p = get_point_by_name((*it).c_str()) + 1;
+        if (p > 0)
+            test_mask |= 1 << p;
+    }
+}

=== modified file 'src/profiler.h'
--- src/profiler.h	2011-10-14 12:41:30 +0000
+++ src/profiler.h	2012-05-02 14:54:19 +0000
@@ -37,6 +37,12 @@
 public:
     typedef int Point;
 
+    enum duration {
+        SUBSET,
+        TOTAL,
+        NUM_DURATIONS
+    };
+
     struct Stats
     {
         uint64_t nsamples;
@@ -44,6 +50,7 @@
         double high;
         double total;
         double average;
+        double squares;
     };
 
     class PointPair
@@ -132,10 +139,15 @@
     bool connect(Point p1, Point p2);
     bool sample_point(Point p);
     bool get_stats(Point p1, Point p2, Stats &stats);
+    bool get_stats(Point p1, Point p2, Stats &stats, enum duration d);
     void reset_point(Point p);
     void reset(void);
     Point get_num_points() { return points_.size(); }
     static uint64_t get_timestamp_us();
+    void accumulator_start();
+    void accumulator_end();
+    void get_accumulated_stats(Stats &stats);
+    void init_test(char *str);
 
     /**
      * Gets the profiler singleton
@@ -147,8 +159,13 @@
     }
 
 private:
+    void sample_point(Point p, enum duration d, uint64_t now);
     bool is_valid_point(Point p);
     std::vector<ProfilerPoint *> points_;
+    Stats accumulated_stats;
+    bool accumulator_active;
+    unsigned int accumulator;
+    unsigned int test_mask;
 };