Merge lp:~didrocks/ubiquity/intall-metrics into lp:ubiquity

[Didier Roche-Tolomelli]

Add telemetry function

Always report install metrics for optional reporting later on.
Tested on both Kubuntu and Ubuntu, with:
  - side-by-side install
  - use biggest free space
  - reinstall over existing installation, keeping files
  - reinstall and replace existing installation
  - full disk used
  - lvm
  - crypto
  - manual
  - oem
  - automatic installation

It should also work in debconf UI mode, but I'm unsure how to test it.

[lp:~didrocks/ubiquity/intall-metrics]

6593. By Jean-Baptiste Lallement on 2018-03-09

[ Sebastien Bacher ]
  * gui/gtk/stepPrepare.ui:
    - updated the non-free-software text to not include the mention to the
      MP3 fluendo codecs since those have been superseeded in the recent
      gstreamer version.

6594. By Jean-Baptiste Lallement on 2018-03-09

[ Sebastien Bacher ]
  * Rename the "Preparing to install Ubuntu" installer step to "Updates and
    other software" according to the design, thanks Matthew for pointing out
    the inconsistency

6595. By Jean-Baptiste Lallement on 2018-03-09

* debian/control: fixed dependency on sensible-utils

[Iain Lane (laney)]

Thanks, this is great. I have some comments inline, nothing too major. :)

After fixing those, I don't think that you'd need to re-test the full matrix: if it works for one it should work equally for all the others.

Just for the record, as jibel said on IRC, it would be good to test if this works with preseeding. Even if not, that could be fixed later on though.

review: Needs Fixing

[Didier Roche-Tolomelli (didrocks)]

Waiting for your comments on my answer before going further :)

[Didier Roche-Tolomelli (didrocks)]

ok, sounds like I can't make it a singleton easily without breaking the MVC semantic in place.

I have changed the global as a class variable and made the time measurement relative + adding a "start time"

Let me know what you think about class singleton vs code consistency and "Steps" naming

[Iain Lane (laney)]

some replies to your comments but I had to do them on the old version of the diff

thanks for considering. As commented I think we should discuss what the requirements for timing data are as this seems a bit strange to me. After getting that settled and maybe adding some protection to the file saving I'm OK with this once jibel has tested and given his thumbs up. Cheers!

[Didier Roche-Tolomelli (didrocks)]

Done of protecting telemetry. Let's wait on finishing our discussion on what to collect :)

[Sebastien Bacher (seb128)]

Ok, I've replied on the "recording the steps" comments, those seem like they could be useful to validate design changes and see their impact on the users

[Didier Roche-Tolomelli (didrocks)]

Tested latest rebased version with the GTK installer, all working!

[Iain Lane (laney)]

ok from my side, waiting for QA to approve too

review: Approve

[Jean-Baptiste Lallement (jibel)]

LGTM too. Thanks!

review: Approve

[lp:~didrocks/ubiquity/intall-metrics]

6596. By Jean-Baptiste Lallement on 2018-03-15

* Added telemetry feature to collect installation metrics and optional reporting later on. Thanks didrocks

=== modified file 'debian/changelog'
--- debian/changelog	2018-03-09 10:43:07 +0000
+++ debian/changelog	2018-03-13 11:08:32 +0000
@@ -12,6 +12,10 @@
     other software" according to the design, thanks Matthew for pointing 
     out the inconsistency
 
+  [ Didier Roche ]
+  * Collect and dump telemetry info as a log file: (LP: #1755456)
+    Those installation telemetry info are for optional upload later on.
+
  -- Mathieu Trudel-Lapierre <[e-mail]>  Wed, 28 Feb 2018 14:36:54 -0500
 
 ubiquity (18.04.3) bionic; urgency=medium

=== modified file 'ubiquity/frontend/debconf_ui.py'
--- ubiquity/frontend/debconf_ui.py	2013-02-19 11:33:07 +0000
+++ ubiquity/frontend/debconf_ui.py	2018-03-13 11:08:32 +0000
@@ -33,7 +33,7 @@
 
 import debconf
 
-from ubiquity import i18n
+from ubiquity import i18n, telemetry
 from ubiquity.components import install, plugininstall
 from ubiquity.frontend.base import BaseFrontend, Controller
 from ubiquity.plugin import Plugin
@@ -94,6 +94,9 @@
                 'cannot continue without them.'), file=sys.stderr)
             sys.exit(1)
 
+        telemetry.get().set_installer_type('DebConf')
+        telemetry.get().add_stage(telemetry.START_INSTALL_STAGE_TAG)
+
         self.pagesindex = 0
         self.pageslen = 0
         self.pages = []
@@ -152,6 +155,7 @@
                             "Install failed with exit code %s; see "
                             "/var/log/syslog" % ret)
 
+            telemetry.get().done(self.db)
             return 0
         else:
             return 10

=== modified file 'ubiquity/frontend/gtk_ui.py'
--- ubiquity/frontend/gtk_ui.py	2017-09-28 21:25:09 +0000
+++ ubiquity/frontend/gtk_ui.py	2018-03-13 11:08:32 +0000
@@ -54,7 +54,7 @@
     from ubiquity import gtkwidgets
 
 from ubiquity import (
-    filteredcommand, gsettings, i18n, validation, misc, osextras)
+    filteredcommand, gsettings, i18n, validation, misc, osextras, telemetry)
 from ubiquity.components import install, plugininstall, partman_commit
 import ubiquity.frontend.base
 from ubiquity.frontend.base import BaseFrontend
@@ -757,6 +757,8 @@
             self.debconf_progress_cancellable(False)
             self.refresh()
 
+        telemetry.get().set_installer_type('GTK')
+
         self.set_current_page(0)
         self.live_installer.show()
 
@@ -821,6 +823,8 @@
         # postinstall will exit here by calling Gtk.main_quit in
         # find_next_step.
 
+        telemetry.get().done(self.db)
+
         self.unlock_environment()
         if self.oem_user_config:
             self.quit_installer()
@@ -1555,6 +1559,7 @@
         for i in range(len(self.pages))[index + 1:]:
             self.dot_grid.get_child_at(i, 0).set_fraction(0)
 
+        telemetry.get().add_stage(name)
         syslog.syslog('switched to page %s' % name)
 
     # Callbacks provided to components.
@@ -1645,6 +1650,8 @@
             return False
 
     def switch_to_install_interface(self):
+        if not self.installing:
+            telemetry.get().add_stage(telemetry.START_INSTALL_STAGE_TAG)
         self.installing = True
         self.lockdown_environment()
 

=== modified file 'ubiquity/frontend/kde_components/PartAuto.py'
--- ubiquity/frontend/kde_components/PartAuto.py	2016-08-26 07:13:37 +0000
+++ ubiquity/frontend/kde_components/PartAuto.py	2018-03-13 11:08:32 +0000
@@ -252,18 +252,20 @@
             disk_id = self.extra_options['use_device'][1][comboText][0]
             disk_id = disk_id.rsplit('/', 1)[1]
             option = self.extra_options['resize'][disk_id][0]
-            return option, '%d B' % self.resizeSize
+            return option, '%d B' % self.resizeSize, 'resize_use_free'
         elif choice == self.useDeviceChoice:
             return (self.extra_options['use_device'][0],
-                    str(self.part_auto_disk_box.currentText()))
+                    str(self.part_auto_disk_box.currentText()), 'use_device')
         elif choice == self.lvm_choice:
             return (choice,
-                    str(self.part_auto_disk_box.currentText()))
+                    str(self.part_auto_disk_box.currentText()), 'use_lvm')
         elif choice == self.crypto_choice:
             return (choice,
-                    str(self.part_auto_disk_box.currentText()))
+                    str(self.part_auto_disk_box.currentText()), 'use_crypto')
+        elif choice == self.manualChoice:
+            return choice, None, 'manual'
         else:
-            return choice, None
+            return choice, None, 'unknown'
 
     def on_disks_combo_changed(self, index):
         for e in self.bar_frames:

=== modified file 'ubiquity/frontend/kde_ui.py'
--- ubiquity/frontend/kde_ui.py	2017-01-09 20:15:33 +0000
+++ ubiquity/frontend/kde_ui.py	2018-03-13 11:08:32 +0000
@@ -41,7 +41,7 @@
 sip.setapi("QVariant", 1)
 from PyQt5 import QtCore, QtGui, QtWidgets, uic
 
-from ubiquity import filteredcommand, i18n, misc
+from ubiquity import filteredcommand, i18n, misc, telemetry
 from ubiquity.components import partman_commit, install, plugininstall
 import ubiquity.frontend.base
 from ubiquity.frontend.base import BaseFrontend
@@ -454,6 +454,8 @@
                 self.get_string('ubiquity/install/title'))
             self.refresh()
 
+        telemetry.get().set_installer_type('KDE')
+
         # Start the interface
         self.set_current_page(0)
 
@@ -529,6 +531,8 @@
             self.start_slideshow()
             self.run_main_loop()
 
+            telemetry.get().done(self.db)
+
             quitText = '<qt>%s</qt>' % self.get_string("finished_label")
             rebootButtonText = self.get_string("reboot_button")
             shutdownButtonText = self.get_string("shutdown_button")
@@ -1121,6 +1125,7 @@
         self.ui.content_widget.show()
         self.current_page = newPageID
         name = self.step_name(newPageID)
+        telemetry.get().add_stage(name)
         syslog.syslog('switched to page %s' % name)
         if 'UBIQUITY_GREETER' in os.environ:
             if name == 'language':
@@ -1291,6 +1296,7 @@
         elif finished_step == 'ubi-partman':
             # Flush changes to the database so that when the parallel db
             # starts, it does so with the most recent changes.
+            telemetry.get().add_stage(telemetry.START_INSTALL_STAGE_TAG)
             self.stop_debconf()
             self.start_debconf()
             self._show_progress_bar(True)

=== modified file 'ubiquity/frontend/noninteractive.py'
--- ubiquity/frontend/noninteractive.py	2014-06-30 13:09:16 +0000
+++ ubiquity/frontend/noninteractive.py	2018-03-13 11:08:32 +0000
@@ -32,7 +32,7 @@
 
 from gi.repository import GLib
 
-from ubiquity import filteredcommand, i18n, misc
+from ubiquity import filteredcommand, i18n, misc, telemetry
 from ubiquity.components import install, plugininstall, partman_commit
 import ubiquity.frontend.base
 from ubiquity.frontend.base import BaseFrontend
@@ -77,6 +77,9 @@
                   file=self.console)
             sys.exit(1)
 
+        telemetry.get().set_installer_type('NonInteractive')
+        telemetry.get().add_stage(telemetry.START_INSTALL_STAGE_TAG)
+
         for x in self.pages:
             if issubclass(x.filter_class, Plugin):
                 ui = x.ui
@@ -110,6 +113,7 @@
         if ret == 0:
             self.run_success_cmd()
             print('Installation complete.', file=self.console)
+            telemetry.get().done(self.db)
             if self.get_reboot():
                 misc.execute("reboot")
         if ret != 0:

=== modified file 'ubiquity/plugins/ubi-partman.py'
--- ubiquity/plugins/ubi-partman.py	2016-12-02 13:20:16 +0000
+++ ubiquity/plugins/ubi-partman.py	2018-03-13 11:08:32 +0000
@@ -25,7 +25,8 @@
 
 import debconf
 
-from ubiquity import misc, osextras, parted_server, plugin, validation
+from ubiquity import (misc, osextras, parted_server, plugin,
+                      telemetry, validation)
 from ubiquity.install_misc import archdetect
 
 
@@ -673,22 +674,24 @@
 
     def get_autopartition_choice(self):
         if self.reuse_partition.get_active():
-            return self.extra_options['reuse'][0][0], None
+            return self.extra_options['reuse'][0][0], None, 'reuse_partition'
 
         if self.replace_partition.get_active():
-            return self.extra_options['replace'][0], None
+            return (self.extra_options['replace'][0], None,
+                    'reinstall_partition')
 
         elif self.custom_partitioning.get_active():
-            return self.extra_options['manual'], None
+            return self.extra_options['manual'], None, 'manual'
 
         elif self.resize_use_free.get_active():
             if 'biggest_free' in self.extra_options:
                 choice = self.extra_options['biggest_free'][0]
-                return choice, None
+                return choice, None, 'resize_use_free'
             else:
                 disk_id = self.get_current_disk_partman_id()
                 choice = self.extra_options['resize'][disk_id][0]
-                return choice, '%s B' % self.resizewidget.get_size()
+                return (choice, '%s B' % self.resizewidget.get_size(),
+                        'resize_use_free')
 
         elif self.use_device.get_active():
             def choose_recipe():
@@ -702,13 +705,15 @@
 
                 if not ((want_crypto and have_crypto) or
                         (want_lvm and have_lvm)):
-                    return self.extra_options['use_device'][0]
+                    return self.extra_options['use_device'][0], 'use_device'
 
                 if want_crypto:
-                    return self.extra_options['some_device_crypto']
+                    return (self.extra_options['some_device_crypto'],
+                            'use_crypto')
 
                 if want_lvm:
-                    return self.extra_options['some_device_lvm']
+                    return (self.extra_options['some_device_lvm'],
+                            'use_lvm')
 
                 # Something went horribly wrong, we should have returned
                 # earlier
@@ -717,9 +722,9 @@
             i = self.part_auto_select_drive.get_active_iter()
             m = self.part_auto_select_drive.get_model()
             disk = m.get_value(i, 0)
-            choice = choose_recipe()
+            choice, method = choose_recipe()
             # Is the encoding necessary?
-            return choice, misc.utf8(disk, errors='replace')
+            return choice, misc.utf8(disk, errors='replace'), method
 
         else:
             raise AssertionError("Couldn't get autopartition choice")
@@ -3249,10 +3254,11 @@
             self.preseed('grub-installer/bootdev', self.ui.get_grub_choice())
 
         if self.current_question.endswith('automatically_partition'):
-            (autopartition_choice, self.extra_choice) = \
+            (autopartition_choice, self.extra_choice, method) = \
                 self.ui.get_autopartition_choice()
             self.preseed_as_c(self.current_question, autopartition_choice,
                               seen=False)
+            telemetry.get().set_partition_method(method)
             # Don't exit partman yet.
         else:
             self.finish_partitioning = True

=== added file 'ubiquity/telemetry.py'
--- ubiquity/telemetry.py	1970-01-01 00:00:00 +0000
+++ ubiquity/telemetry.py	2018-03-13 11:08:32 +0000
@@ -0,0 +1,96 @@
+# -*- coding: utf-8; Mode: Python; indent-tabs-mode: nil; tab-width: 4 -*-
+
+# Copyright (C) 2018 Canonical Ltd.
+#
+# Functions useful for the final install.py script and for ubiquity
+# plugins to use
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+
+
+import json
+import os
+import stat
+import syslog
+import time
+
+from ubiquity.misc import raise_privileges
+
+START_INSTALL_STAGE_TAG = 'start_install'
+
+
+def get():
+    """Return a singleton _Telemetry instance."""
+    if _Telemetry._telemetry is None:
+        _Telemetry._telemetry = _Telemetry()
+    return _Telemetry._telemetry
+
+
+class _Telemetry():
+
+    _telemetry = None
+
+    def __init__(self):
+        self._metrics = {}
+        self._stages_hist = {}
+        self._start_time = time.time()
+        self.add_stage('start')
+        self._dest_path = '/target/var/log/installer/telemetry'
+        try:
+            with open('/cdrom/.disk/info') as f:
+                self._metrics['Media'] = f.readline()
+        except FileNotFoundError:
+            self._metrics['Media'] = 'unknown'
+
+    def add_stage(self, stage_name):
+        """Record installer stage with current time"""
+        self._stages_hist[int(time.time() - self._start_time)] = stage_name
+
+    def set_installer_type(self, installer_type):
+        """Record installer type"""
+        self._metrics['Type'] = installer_type
+
+    def set_partition_method(self, method):
+        """Record anynomized partition method"""
+        self._metrics['PartitionMethod'] = method
+
+    @raise_privileges
+    def done(self, db):
+        """Close telemetry collection
+
+        Set as installation done, add additional info and save to
+        destination file"""
+        self.add_stage('done')
+
+        self._metrics['DownloadUpdates'] = db.get('ubiquity/download_updates')
+        self._metrics['Language'] = db.get('localechooser/languagelist')
+        self._metrics['Minimal'] = db.get('ubiquity/minimal_install')
+        self._metrics['RestrictedAddons'] = db.get('ubiquity/use_nonfree')
+        self._metrics['Stages'] = self._stages_hist
+
+        target_dir = os.path.dirname(self._dest_path)
+        try:
+            if not os.path.exists(target_dir):
+                os.makedirs(target_dir)
+            with open(self._dest_path, 'w') as f:
+                json.dump(self._metrics, f)
+            os.chmod(self._dest_path,
+                     stat.S_IRUSR | stat.S_IWUSR |
+                     stat.S_IRGRP | stat.S_IROTH)
+        except OSError as e:
+            syslog.syslog(syslog.LOG_ERR,
+                          "Exception while storing telemetry data: " + str(e))
+
+# vim:ai:et:sts=4:tw=80:sw=4: