Merge lp:~didrocks/ubuntu-release-upgrader/add_telemetry into lp:ubuntu-release-upgrader

Proposed by Didier Roche on 2018-05-04
Status: Merged
Approved by: Didier Roche on 2018-05-29
Approved revision: 3135
Merged at revision: 3131
Proposed branch: lp:~didrocks/ubuntu-release-upgrader/add_telemetry
Merge into: lp:ubuntu-release-upgrader
Diff against target: 549 lines (+176/-55)
8 files modified
DistUpgrade/DistUpgradeController.py (+16/-20)
DistUpgrade/DistUpgradeView.py (+11/-7)
DistUpgrade/DistUpgradeViewGtk3.py (+20/-16)
DistUpgrade/DistUpgradeViewKDE.py (+14/-10)
DistUpgrade/DistUpgradeViewNonInteractive.py (+3/-0)
DistUpgrade/DistUpgradeViewText.py (+5/-2)
DistUpgrade/telemetry.py (+101/-0)
debian/changelog (+6/-0)
To merge this branch: bzr merge lp:~didrocks/ubuntu-release-upgrader/add_telemetry
Reviewer Review Type Date Requested Status
Didier Roche Approve on 2018-05-29
Brian Murray 2018-05-04 Approve on 2018-05-23
Review via email: mp+345088@code.launchpad.net

Commit message

Add upgrade telemetry data for later collect in ubuntu-report.

Description of the change

Add upgrade telemetry data for later collect in ubuntu-report.

This was tested with 17.04 -> 18.04 upgrade, in text mode, GTK3 and KDE.

To post a comment you must log in.
Brian Murray (brian-murray) wrote :

Keep in mind that it is possible to from one release to multiple releases e.g. from xenial to artful or bionic. So without knowing the database structure I think it would make sense to also record the "To" release. You can find this in DistUpgradeController.py.

I'm also curious about how this information will be used because the use cases might benefit from gathering additional information like the version of release upgrader being used or environmental variables set.

Otherwise there are a couple of typos and one nitpick to have a look at.

review: Needs Information
Didier Roche (didrocks) wrote :

Thanks for the review!
There isn't a fixed database structure schema. However, keep in mind that ubuntu-report gather this results (as well as installers one) under some subfields on the json, alongside collecting a lot of others information.
So basically, we have the "To" and a lot of other details. Moreinfo on https://github.com/ubuntu/ubuntu-report.

For now, we collect those details, similarly than for ubiquity, and we'll organize them next cycle if nobody on IS step up first.

Typos fixed and pushed. Thanks!

3132. By Didier Roche on 2018-05-11

Fix some typos

3133. By Didier Roche on 2018-05-18

Use uptime time instead of time.time(), which is sensitive to system
clock reset by NTP or others…

3134. By Didier Roche on 2018-05-18

Fetch if users enabled third party repos before upgrade

3135. By Didier Roche on 2018-05-18

Fetch original install media

Brian Murray (brian-murray) wrote :

This looks good to me to merge and upload after the one change is made.

While this may be more useful for the failure case you might consider including what components (-updates, -security, main, universe, etc...) are enabled on the system that was upgraded. Here's an example from an upgrade log file.

2018-05-12 20:32:36,997 DEBUG found components: {'bionic-updates': {'main', 'universe', 'multiverse', 'restricted'}, 'bionic-security': {'main', 'universe', 'multiverse', 'restricted'}, 'bionic': {'main', 'universe', 'multiverse', 'restricted'}}

Didier Roche (didrocks) wrote :

We don't want to include the enabled components for now, but thanks for the suggestion! I'll fix the current uptime comment.

Thanks again for the review :)

review: Approve
Didier Roche (didrocks) wrote :

Tested artful -> bionic upgrade with this code and it reports the expected telemetry data.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'DistUpgrade/DistUpgradeController.py'
--- DistUpgrade/DistUpgradeController.py 2018-03-16 17:56:15 +0000
+++ DistUpgrade/DistUpgradeController.py 2018-05-18 10:01:29 +0000
@@ -33,6 +33,7 @@
33import copy33import copy
34from configparser import NoOptionError34from configparser import NoOptionError
35from configparser import ConfigParser as SafeConfigParser35from configparser import ConfigParser as SafeConfigParser
36from .telemetry import get as get_telemetry
36from .utils import (country_mirror,37from .utils import (country_mirror,
37 url_downloadable,38 url_downloadable,
38 check_and_fix_xbit,39 check_and_fix_xbit,
@@ -45,14 +46,7 @@
45from string import Template46from string import Template
46from urllib.parse import urlsplit47from urllib.parse import urlsplit
4748
48from .DistUpgradeView import (49from .DistUpgradeView import Step
49 STEP_PREPARE,
50 STEP_MODIFY_SOURCES,
51 STEP_FETCH,
52 STEP_INSTALL,
53 STEP_CLEANUP,
54 STEP_REBOOT,
55)
56from .DistUpgradeCache import MyCache50from .DistUpgradeCache import MyCache
57from .DistUpgradeConfigParser import DistUpgradeConfig51from .DistUpgradeConfigParser import DistUpgradeConfig
58from .DistUpgradeQuirks import DistUpgradeQuirks52from .DistUpgradeQuirks import DistUpgradeQuirks
@@ -866,6 +860,7 @@
866 "'software-properties' tool or "860 "'software-properties' tool or "
867 "your package manager."861 "your package manager."
868 ))862 ))
863 get_telemetry().set_using_third_party_sources(self.sources_disabled)
869 return True864 return True
870865
871 def _logChanges(self):866 def _logChanges(self):
@@ -1760,7 +1755,7 @@
1760 def fullUpgrade(self):1755 def fullUpgrade(self):
1761 # sanity check (check for ubuntu-desktop, brokenCache etc)1756 # sanity check (check for ubuntu-desktop, brokenCache etc)
1762 self._view.updateStatus(_("Checking package manager"))1757 self._view.updateStatus(_("Checking package manager"))
1763 self._view.setStep(STEP_PREPARE)1758 self._view.setStep(Step.PREPARE)
17641759
1765 if not self.prepare():1760 if not self.prepare():
1766 logging.error("self.prepare() failed")1761 logging.error("self.prepare() failed")
@@ -1829,7 +1824,7 @@
1829 self.abort()1824 self.abort()
18301825
1831 # update sources.list1826 # update sources.list
1832 self._view.setStep(STEP_MODIFY_SOURCES)1827 self._view.setStep(Step.MODIFY_SOURCES)
1833 self._view.updateStatus(_("Updating repository information"))1828 self._view.updateStatus(_("Updating repository information"))
1834 if not self.updateSourcesList():1829 if not self.updateSourcesList():
1835 self.abort()1830 self.abort()
@@ -1901,14 +1896,14 @@
1901 self.abort()1896 self.abort()
19021897
1903 # fetch the stuff1898 # fetch the stuff
1904 self._view.setStep(STEP_FETCH)1899 self._view.setStep(Step.FETCH)
1905 self._view.updateStatus(_("Fetching"))1900 self._view.updateStatus(_("Fetching"))
1906 if not self.doDistUpgradeFetching():1901 if not self.doDistUpgradeFetching():
1907 self._enableAptCronJob()1902 self._enableAptCronJob()
1908 self.abort()1903 self.abort()
19091904
1910 # now do the upgrade1905 # now do the upgrade
1911 self._view.setStep(STEP_INSTALL)1906 self._view.setStep(Step.INSTALL)
1912 self._view.updateStatus(_("Upgrading"))1907 self._view.updateStatus(_("Upgrading"))
1913 if not self.doDistUpgrade():1908 if not self.doDistUpgrade():
1914 # run the post install scripts (for stuff like UUID conversion)1909 # run the post install scripts (for stuff like UUID conversion)
@@ -1922,7 +1917,7 @@
1922 sys.exit(1)1917 sys.exit(1)
19231918
1924 # do post-upgrade stuff1919 # do post-upgrade stuff
1925 self._view.setStep(STEP_CLEANUP)1920 self._view.setStep(Step.CLEANUP)
1926 self._view.updateStatus(_("Searching for obsolete software"))1921 self._view.updateStatus(_("Searching for obsolete software"))
1927 self.doPostUpgrade()1922 self.doPostUpgrade()
19281923
@@ -1935,8 +1930,9 @@
1935 os.unlink("/var/lib/ubuntu-release-upgrader/release-upgrade-available")1930 os.unlink("/var/lib/ubuntu-release-upgrader/release-upgrade-available")
19361931
1937 # done, ask for reboot1932 # done, ask for reboot
1938 self._view.setStep(STEP_REBOOT)1933 self._view.setStep(Step.REBOOT)
1939 self._view.updateStatus(_("System upgrade is complete.")) 1934 self._view.updateStatus(_("System upgrade is complete."))
1935 get_telemetry().done()
1940 # FIXME should we look into /var/run/reboot-required here?1936 # FIXME should we look into /var/run/reboot-required here?
1941 if (not inside_chroot() and1937 if (not inside_chroot() and
1942 self._view.confirmRestart()):1938 self._view.confirmRestart()):
@@ -1950,20 +1946,20 @@
1950 1946
1951 def doPartialUpgrade(self):1947 def doPartialUpgrade(self):
1952 " partial upgrade mode, useful for repairing "1948 " partial upgrade mode, useful for repairing "
1953 self._view.setStep(STEP_PREPARE)1949 self._view.setStep(Step.PREPARE)
1954 self._view.hideStep(STEP_MODIFY_SOURCES)1950 self._view.hideStep(Step.MODIFY_SOURCES)
1955 self._view.hideStep(STEP_REBOOT)1951 self._view.hideStep(Step.REBOOT)
1956 self._partialUpgrade = True1952 self._partialUpgrade = True
1957 self.prepare()1953 self.prepare()
1958 if not self.doPostInitialUpdate():1954 if not self.doPostInitialUpdate():
1959 return False1955 return False
1960 if not self.askDistUpgrade():1956 if not self.askDistUpgrade():
1961 return False1957 return False
1962 self._view.setStep(STEP_FETCH)1958 self._view.setStep(Step.FETCH)
1963 self._view.updateStatus(_("Fetching"))1959 self._view.updateStatus(_("Fetching"))
1964 if not self.doDistUpgradeFetching():1960 if not self.doDistUpgradeFetching():
1965 return False1961 return False
1966 self._view.setStep(STEP_INSTALL)1962 self._view.setStep(Step.INSTALL)
1967 self._view.updateStatus(_("Upgrading"))1963 self._view.updateStatus(_("Upgrading"))
1968 if not self.doDistUpgrade():1964 if not self.doDistUpgrade():
1969 self._view.information(_("Upgrade complete"),1965 self._view.information(_("Upgrade complete"),
@@ -1971,7 +1967,7 @@
1971 "were errors during the upgrade "1967 "were errors during the upgrade "
1972 "process."))1968 "process."))
1973 return False1969 return False
1974 self._view.setStep(STEP_CLEANUP)1970 self._view.setStep(Step.CLEANUP)
1975 if not self.doPostUpgrade():1971 if not self.doPostUpgrade():
1976 self._view.information(_("Upgrade complete"),1972 self._view.information(_("Upgrade complete"),
1977 _("The upgrade has completed but there "1973 _("The upgrade has completed but there "
19781974
=== modified file 'DistUpgrade/DistUpgradeView.py'
--- DistUpgrade/DistUpgradeView.py 2017-05-18 17:33:48 +0000
+++ DistUpgrade/DistUpgradeView.py 2018-05-18 10:01:29 +0000
@@ -21,7 +21,9 @@
2121
22from .DistUpgradeGettext import gettext as _22from .DistUpgradeGettext import gettext as _
23from .DistUpgradeGettext import ngettext23from .DistUpgradeGettext import ngettext
24from .telemetry import get as get_telemetry
24import apt25import apt
26from enum import Enum
25import errno27import errno
26import os28import os
27import apt_pkg 29import apt_pkg
@@ -252,13 +254,14 @@
252 def hide(self):254 def hide(self):
253 pass255 pass
254256
255(STEP_PREPARE,257class Step(Enum):
256 STEP_MODIFY_SOURCES,258 PREPARE = 1
257 STEP_FETCH,259 MODIFY_SOURCES = 2
258 STEP_INSTALL,260 FETCH = 3
259 STEP_CLEANUP,261 INSTALL = 4
260 STEP_REBOOT,262 CLEANUP = 5
261 STEP_N) = range(1,8)263 REBOOT = 6
264 N = 7
262265
263# Declare these translatable strings from the .ui files here so that266# Declare these translatable strings from the .ui files here so that
264# xgettext picks them up.267# xgettext picks them up.
@@ -304,6 +307,7 @@
304 4. Post upgrade stuff307 4. Post upgrade stuff
305 5. Complete308 5. Complete
306 """309 """
310 get_telemetry().add_stage(step.name)
307 pass311 pass
308 def hideStep(self, step):312 def hideStep(self, step):
309 " hide a certain step from the GUI "313 " hide a certain step from the GUI "
310314
=== modified file 'DistUpgrade/DistUpgradeViewGtk3.py'
--- DistUpgrade/DistUpgradeViewGtk3.py 2016-02-23 17:06:42 +0000
+++ DistUpgrade/DistUpgradeViewGtk3.py 2018-05-18 10:01:29 +0000
@@ -52,6 +52,7 @@
52from .DistUpgradeApport import run_apport, apport_crash52from .DistUpgradeApport import run_apport, apport_crash
5353
54from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress54from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress
55from .telemetry import get as get_telemetry
55from .SimpleGtk3builderApp import SimpleGtkbuilderApp56from .SimpleGtk3builderApp import SimpleGtkbuilderApp
5657
57import gettext58import gettext
@@ -70,7 +71,7 @@
70 """ update is called regularly so that the gui can be redrawn """71 """ update is called regularly so that the gui can be redrawn """
71 if text:72 if text:
72 self.status.set_text(text)73 self.status.set_text(text)
73 self.progress.set_fraction(step/float(self.totalSteps))74 self.progress.set_fraction(step.value/float(self.totalSteps))
74 while Gtk.events_pending():75 while Gtk.events_pending():
75 Gtk.main_iteration()76 Gtk.main_iteration()
76 def ask_cdrom_name(self):77 def ask_cdrom_name(self):
@@ -452,6 +453,8 @@
452 # check if we have a display etc453 # check if we have a display etc
453 Gtk.init_check(sys.argv)454 Gtk.init_check(sys.argv)
454455
456 get_telemetry().set_updater_type('GTK')
457
455 try:458 try:
456 locale.bindtextdomain("ubuntu-release-upgrader",localedir)459 locale.bindtextdomain("ubuntu-release-upgrader",localedir)
457 gettext.textdomain("ubuntu-release-upgrader")460 gettext.textdomain("ubuntu-release-upgrader")
@@ -472,7 +475,7 @@
472 # terminal stuff475 # terminal stuff
473 self.create_terminal()476 self.create_terminal()
474477
475 self.prev_step = 0 # keep a record of the latest step478 self.prev_step = None # keep a record of the latest step
476 # we don't use this currently479 # we don't use this currently
477 #self.window_main.set_keep_above(True)480 #self.window_main.set_keep_above(True)
478 self.icontheme = Gtk.IconTheme.get_default()481 self.icontheme = Gtk.IconTheme.get_default()
@@ -598,44 +601,45 @@
598 def updateStatus(self, msg):601 def updateStatus(self, msg):
599 self.label_status.set_text("%s" % msg)602 self.label_status.set_text("%s" % msg)
600 def hideStep(self, step):603 def hideStep(self, step):
601 image = getattr(self,"image_step%i" % step)604 image = getattr(self,"image_step%i" % step.value)
602 label = getattr(self,"label_step%i" % step)605 label = getattr(self,"label_step%i" % step.value)
603 #arrow = getattr(self,"arrow_step%i" % step)606 #arrow = getattr(self,"arrow_step%i" % step.value)
604 image.hide()607 image.hide()
605 label.hide()608 label.hide()
606 def showStep(self, step):609 def showStep(self, step):
607 image = getattr(self,"image_step%i" % step)610 image = getattr(self,"image_step%i" % step.value)
608 label = getattr(self,"label_step%i" % step)611 label = getattr(self,"label_step%i" % step.value)
609 image.show()612 image.show()
610 label.show()613 label.show()
611 def abort(self):614 def abort(self):
612 size = Gtk.IconSize.MENU615 size = Gtk.IconSize.MENU
613 step = self.prev_step616 step = self.prev_step
614 if step > 0:617 if step:
615 image = getattr(self,"image_step%i" % step)618 image = getattr(self,"image_step%i" % step.value)
616 arrow = getattr(self,"arrow_step%i" % step)619 arrow = getattr(self,"arrow_step%i" % step.value)
617 image.set_from_stock(Gtk.STOCK_CANCEL, size)620 image.set_from_stock(Gtk.STOCK_CANCEL, size)
618 image.show()621 image.show()
619 arrow.hide()622 arrow.hide()
620 def setStep(self, step):623 def setStep(self, step):
624 super(DistUpgradeViewGtk3, self).setStep(step)
621 if self.icontheme.rescan_if_needed():625 if self.icontheme.rescan_if_needed():
622 logging.debug("icon theme changed, re-reading")626 logging.debug("icon theme changed, re-reading")
623 # first update the "previous" step as completed627 # first update the "previous" step as completed
624 size = Gtk.IconSize.MENU628 size = Gtk.IconSize.MENU
625 attrlist=Pango.AttrList()629 attrlist=Pango.AttrList()
626 if self.prev_step:630 if self.prev_step:
627 image = getattr(self,"image_step%i" % self.prev_step)631 image = getattr(self,"image_step%i" % self.prev_step.value)
628 label = getattr(self,"label_step%i" % self.prev_step)632 label = getattr(self,"label_step%i" % self.prev_step.value)
629 arrow = getattr(self,"arrow_step%i" % self.prev_step)633 arrow = getattr(self,"arrow_step%i" % self.prev_step.value)
630 label.set_property("attributes",attrlist)634 label.set_property("attributes",attrlist)
631 image.set_from_stock(Gtk.STOCK_APPLY, size)635 image.set_from_stock(Gtk.STOCK_APPLY, size)
632 image.show()636 image.show()
633 arrow.hide()637 arrow.hide()
634 self.prev_step = step638 self.prev_step = step
635 # show the an arrow for the current step and make the label bold639 # show the an arrow for the current step and make the label bold
636 image = getattr(self,"image_step%i" % step)640 image = getattr(self,"image_step%i" % step.value)
637 label = getattr(self,"label_step%i" % step)641 label = getattr(self,"label_step%i" % step.value)
638 arrow = getattr(self,"arrow_step%i" % step)642 arrow = getattr(self,"arrow_step%i" % step.value)
639 # check if that step was not hidden with hideStep()643 # check if that step was not hidden with hideStep()
640 if not label.get_property("visible"):644 if not label.get_property("visible"):
641 return645 return
642646
=== modified file 'DistUpgrade/DistUpgradeViewKDE.py'
--- DistUpgrade/DistUpgradeViewKDE.py 2018-01-22 11:12:39 +0000
+++ DistUpgrade/DistUpgradeViewKDE.py 2018-05-18 10:01:29 +0000
@@ -62,6 +62,7 @@
62from .DistUpgradeApport import run_apport, apport_crash62from .DistUpgradeApport import run_apport, apport_crash
6363
64from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress64from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress
65from .telemetry import get as get_telemetry
6566
66import select67import select
67import gettext68import gettext
@@ -251,7 +252,7 @@
251 """ update is called regularly so that the gui can be redrawn """252 """ update is called regularly so that the gui can be redrawn """
252 if text:253 if text:
253 self.status.setText(text)254 self.status.setText(text)
254 self.progressbar.setValue(step/float(self.totalSteps))255 self.progressbar.setValue(step.value/float(self.totalSteps))
255 QApplication.processEvents()256 QApplication.processEvents()
256257
257 def ask_cdrom_name(self):258 def ask_cdrom_name(self):
@@ -560,6 +561,8 @@
560 """KDE frontend of the distUpgrade tool"""561 """KDE frontend of the distUpgrade tool"""
561 def __init__(self, datadir=None, logdir=None):562 def __init__(self, datadir=None, logdir=None):
562 DistUpgradeView.__init__(self)563 DistUpgradeView.__init__(self)
564
565 get_telemetry().set_updater_type('KDE')
563 # silence the PyQt4 logger566 # silence the PyQt4 logger
564 logger = logging.getLogger("PyQt4")567 logger = logging.getLogger("PyQt4")
565 logger.setLevel(logging.INFO)568 logger.setLevel(logging.INFO)
@@ -613,7 +616,7 @@
613 self.window_main.setParent(self)616 self.window_main.setParent(self)
614 self.window_main.show()617 self.window_main.show()
615618
616 self.prev_step = 0 # keep a record of the latest step619 self.prev_step = None # keep a record of the latest step
617620
618 self._opCacheProgress = KDEOpProgress(self.window_main.progressbar_cache, self.window_main.progress_text)621 self._opCacheProgress = KDEOpProgress(self.window_main.progressbar_cache, self.window_main.progress_text)
619 self._acquireProgress = KDEAcquireProgressAdapter(self)622 self._acquireProgress = KDEAcquireProgressAdapter(self)
@@ -733,15 +736,15 @@
733 self.window_main.label_status.setText(msg)736 self.window_main.label_status.setText(msg)
734737
735 def hideStep(self, step):738 def hideStep(self, step):
736 image = getattr(self.window_main,"image_step%i" % step)739 image = getattr(self.window_main,"image_step%i" % step.value)
737 label = getattr(self.window_main,"label_step%i" % step)740 label = getattr(self.window_main,"label_step%i" % step.value)
738 image.hide()741 image.hide()
739 label.hide()742 label.hide()
740743
741 def abort(self):744 def abort(self):
742 step = self.prev_step745 step = self.prev_step
743 if step > 0:746 if step:
744 image = getattr(self.window_main,"image_step%i" % step)747 image = getattr(self.window_main,"image_step%i" % step.value)
745 cancelIcon = _icon("dialog-cancel",748 cancelIcon = _icon("dialog-cancel",
746 fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-cancel.png",749 fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-cancel.png",
747 "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-cancel.png",750 "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-cancel.png",
@@ -750,6 +753,7 @@
750 image.show()753 image.show()
751754
752 def setStep(self, step):755 def setStep(self, step):
756 super(DistUpgradeViewKDE , self).setStep(step)
753 okIcon = _icon("dialog-ok",757 okIcon = _icon("dialog-ok",
754 fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-ok.png",758 fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-ok.png",
755 "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-ok.png",759 "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-ok.png",
@@ -760,15 +764,15 @@
760 "/usr/share/icons/crystalsvg/16x16/actions/1rightarrow.png"])764 "/usr/share/icons/crystalsvg/16x16/actions/1rightarrow.png"])
761765
762 if self.prev_step:766 if self.prev_step:
763 image = getattr(self.window_main,"image_step%i" % self.prev_step)767 image = getattr(self.window_main,"image_step%i" % self.prev_step.value)
764 label = getattr(self.window_main,"label_step%i" % self.prev_step)768 label = getattr(self.window_main,"label_step%i" % self.prev_step.value)
765 image.setPixmap(okIcon.pixmap(16, 16))769 image.setPixmap(okIcon.pixmap(16, 16))
766 image.show()770 image.show()
767 ##arrow.hide()771 ##arrow.hide()
768 self.prev_step = step772 self.prev_step = step
769 # show the an arrow for the current step and make the label bold773 # show the an arrow for the current step and make the label bold
770 image = getattr(self.window_main,"image_step%i" % step)774 image = getattr(self.window_main,"image_step%i" % step.value)
771 label = getattr(self.window_main,"label_step%i" % step)775 label = getattr(self.window_main,"label_step%i" % step.value)
772 image.setPixmap(arrowIcon.pixmap(16, 16))776 image.setPixmap(arrowIcon.pixmap(16, 16))
773 image.show()777 image.show()
774 label.setText("<b>" + label.text() + "</b>")778 label.setText("<b>" + label.text() + "</b>")
775779
=== modified file 'DistUpgrade/DistUpgradeViewNonInteractive.py'
--- DistUpgrade/DistUpgradeViewNonInteractive.py 2016-10-10 18:05:08 +0000
+++ DistUpgrade/DistUpgradeViewNonInteractive.py 2018-05-18 10:01:29 +0000
@@ -36,6 +36,7 @@
36from subprocess import PIPE, Popen36from subprocess import PIPE, Popen
3737
38from .DistUpgradeView import DistUpgradeView, InstallProgress, AcquireProgress38from .DistUpgradeView import DistUpgradeView, InstallProgress, AcquireProgress
39from .telemetry import get as get_telemetry
39from .DistUpgradeConfigParser import DistUpgradeConfig40from .DistUpgradeConfigParser import DistUpgradeConfig
4041
4142
@@ -251,6 +252,7 @@
251 " non-interactive version of the upgrade view "252 " non-interactive version of the upgrade view "
252 def __init__(self, datadir=None, logdir=None):253 def __init__(self, datadir=None, logdir=None):
253 DistUpgradeView.__init__(self)254 DistUpgradeView.__init__(self)
255 get_telemetry().set_updater_type('NonInteractive')
254 self.config = DistUpgradeConfig(".")256 self.config = DistUpgradeConfig(".")
255 self._acquireProgress = NonInteractiveAcquireProgress()257 self._acquireProgress = NonInteractiveAcquireProgress()
256 self._installProgress = NonInteractiveInstallProgress(logdir)258 self._installProgress = NonInteractiveInstallProgress(logdir)
@@ -286,6 +288,7 @@
286 4. Post upgrade stuff288 4. Post upgrade stuff
287 5. Complete289 5. Complete
288 """290 """
291 super(DistUpgradeViewNonInteractive, self).setStep(step)
289 pass292 pass
290 def confirmChanges(self, summary, changes, demotions, downloadSize,293 def confirmChanges(self, summary, changes, demotions, downloadSize,
291 actions=None, removal_bold=True):294 actions=None, removal_bold=True):
292295
=== modified file 'DistUpgrade/DistUpgradeViewText.py'
--- DistUpgrade/DistUpgradeViewText.py 2016-04-22 19:33:04 +0000
+++ DistUpgrade/DistUpgradeViewText.py 2018-05-18 10:01:29 +0000
@@ -36,6 +36,7 @@
36 ENCODING,36 ENCODING,
37 InstallProgress,37 InstallProgress,
38 )38 )
39from .telemetry import get as get_telemetry
39import apt.progress40import apt.progress
4041
41import gettext42import gettext
@@ -88,7 +89,7 @@
88 def update(self, text, step):89 def update(self, text, step):
89 """ update is called regularly so that the gui can be redrawn """90 """ update is called regularly so that the gui can be redrawn """
90 if text:91 if text:
91 print("%s (%f)" % (text, step/float(self.totalSteps)*100))92 print("%s (%f)" % (text, step.value/float(self.totalSteps)*100))
92 def ask_cdrom_name(self):93 def ask_cdrom_name(self):
93 return (False, "")94 return (False, "")
94 def change_cdrom(self):95 def change_cdrom(self):
@@ -101,6 +102,7 @@
101 def __init__(self, datadir=None, logdir=None):102 def __init__(self, datadir=None, logdir=None):
102 # indicate that we benefit from using gnu screen103 # indicate that we benefit from using gnu screen
103 self.needs_screen = True104 self.needs_screen = True
105 get_telemetry().set_updater_type('Text')
104 # its important to have a debconf frontend for106 # its important to have a debconf frontend for
105 # packages like "quagga"107 # packages like "quagga"
106 if "DEBIAN_FRONTEND" not in os.environ:108 if "DEBIAN_FRONTEND" not in os.environ:
@@ -116,7 +118,7 @@
116 except Exception as e:118 except Exception as e:
117 logging.warning("Error setting locales (%s)" % e)119 logging.warning("Error setting locales (%s)" % e)
118120
119 self.last_step = 0 # keep a record of the latest step121 self.last_step = None # keep a record of the latest step
120 self._opCacheProgress = apt.progress.text.OpProgress()122 self._opCacheProgress = apt.progress.text.OpProgress()
121 self._acquireProgress = TextAcquireProgress()123 self._acquireProgress = TextAcquireProgress()
122 self._cdromProgress = TextCdromProgressAdapter()124 self._cdromProgress = TextCdromProgressAdapter()
@@ -162,6 +164,7 @@
162 print()164 print()
163 print(_("Aborting"))165 print(_("Aborting"))
164 def setStep(self, step):166 def setStep(self, step):
167 super(DistUpgradeViewText, self).setStep(step)
165 self.last_step = step168 self.last_step = step
166 def showDemotions(self, summary, msg, demotions):169 def showDemotions(self, summary, msg, demotions):
167 self.information(summary, msg, 170 self.information(summary, msg,
168171
=== added file 'DistUpgrade/telemetry.py'
--- DistUpgrade/telemetry.py 1970-01-01 00:00:00 +0000
+++ DistUpgrade/telemetry.py 2018-05-18 10:01:29 +0000
@@ -0,0 +1,101 @@
1# -*- coding: utf-8; Mode: Python; indent-tabs-mode: nil; tab-width: 4 -*-
2
3# Copyright (C) 2018 Canonical Ltd.
4#
5# Functions useful for the final install.py script and for ubiquity
6# plugins to use
7#
8# This program is free software; you can redistribute it and/or modify
9# it under the terms of the GNU General Public License as published by
10# the Free Software Foundation; either version 2 of the License, or
11# (at your option) any later version.
12#
13# This program is distributed in the hope that it will be useful,
14# but WITHOUT ANY WARRANTY; without even the implied warranty of
15# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16# GNU General Public License for more details.
17#
18# You should have received a copy of the GNU General Public License
19# along with this program; if not, write to the Free Software
20# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
21
22
23import logging
24import json
25import os
26import stat
27import subprocess
28
29
30def get():
31 """Return a singleton _Telemetry instance."""
32 if _Telemetry._telemetry is None:
33 _Telemetry._telemetry = _Telemetry()
34 return _Telemetry._telemetry
35
36
37class _Telemetry():
38
39 _telemetry = None
40
41 def __init__(self):
42 self._metrics = {}
43 self._stages_hist = {}
44 self._start_time = self._get_current_uptime()
45 self._metrics["From"] = subprocess.Popen(
46 ["lsb_release", "-r", "-s"], stdout=subprocess.PIPE,
47 universal_newlines=True).communicate()[0].strip()
48 self.add_stage('start')
49 self._dest_path = '/var/log/upgrade/telemetry'
50 try:
51 with open('/var/log/installer/media-info') as f:
52 self._metrics['InstallMedia'] = f.readline()
53 except FileNotFoundError:
54 pass
55
56 def _get_current_uptime(self):
57 """Get current uptime info. None if we couldn't fetch it."""
58 uptime = None
59 try:
60 with open('/proc/uptime') as f:
61 uptime = float(f.read().split()[0])
62 except (FileNotFoundError, OSError, ValueError) as e:
63 logging.warning("Exception while fetching current uptime: " + str(e))
64 return uptime
65
66 def add_stage(self, stage_name):
67 """Record installer stage with current time"""
68 now = self._get_current_uptime()
69 if self._start_time is None or now is None:
70 return
71 self._stages_hist[int(now-self._start_time)] = stage_name
72
73 def set_updater_type(self, updater_type):
74 """Record updater type"""
75 self._metrics['Type'] = updater_type
76
77 def set_using_third_party_sources(self, using):
78 """Record if the user had third party sources"""
79 self._metrics['ThirdPartySources'] = using
80
81 def done(self):
82 """Close telemetry collection
83
84 Save to destination file"""
85
86 self._metrics['Stages'] = self._stages_hist
87
88 target_dir = os.path.dirname(self._dest_path)
89 try:
90 if not os.path.exists(target_dir):
91 os.makedirs(target_dir)
92 with open(self._dest_path, 'w') as f:
93 json.dump(self._metrics, f)
94 os.chmod(self._dest_path,
95 stat.S_IRUSR | stat.S_IWUSR |
96 stat.S_IRGRP | stat.S_IROTH)
97 except OSError as e:
98 logging.warning("Exception while storing telemetry data: " +
99 str(e))
100
101# vim:ai:et:sts=4:tw=80:sw=4:
0102
=== modified file 'debian/changelog'
--- debian/changelog 2018-05-02 19:43:50 +0000
+++ debian/changelog 2018-05-18 10:01:29 +0000
@@ -1,3 +1,9 @@
1ubuntu-release-upgrader (1:18.04.19) UNRELEASED; urgency=medium
2
3 * Add upgrade telemetry information (LP: #1755456)
4
5 -- Didier Roche <didrocks@ubuntu.com> Thu, 03 May 2018 15:55:34 +0200
6
1ubuntu-release-upgrader (1:18.04.18) bionic; urgency=medium7ubuntu-release-upgrader (1:18.04.18) bionic; urgency=medium
28
3 * data/removal_blacklist.cfg: Drop unity, ubuntu-gnome-desktop, and9 * data/removal_blacklist.cfg: Drop unity, ubuntu-gnome-desktop, and

Subscribers

People subscribed via source and target branches