Merge lp:~didrocks/ubuntu-release-upgrader/add_telemetry into lp:ubuntu-release-upgrader
- add_telemetry
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Didier Roche-Tolomelli |
Approved revision: | 3135 |
Merged at revision: | 3131 |
Proposed branch: | lp:~didrocks/ubuntu-release-upgrader/add_telemetry |
Merge into: | lp:ubuntu-release-upgrader |
Diff against target: |
549 lines (+176/-55) 8 files modified
DistUpgrade/DistUpgradeController.py (+16/-20) DistUpgrade/DistUpgradeView.py (+11/-7) DistUpgrade/DistUpgradeViewGtk3.py (+20/-16) DistUpgrade/DistUpgradeViewKDE.py (+14/-10) DistUpgrade/DistUpgradeViewNonInteractive.py (+3/-0) DistUpgrade/DistUpgradeViewText.py (+5/-2) DistUpgrade/telemetry.py (+101/-0) debian/changelog (+6/-0) |
To merge this branch: | bzr merge lp:~didrocks/ubuntu-release-upgrader/add_telemetry |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Didier Roche-Tolomelli | Approve | ||
Brian Murray | Approve | ||
Review via email: mp+345088@code.launchpad.net |
Commit message
Add upgrade telemetry data for later collect in ubuntu-report.
Description of the change
Add upgrade telemetry data for later collect in ubuntu-report.
This was tested with 17.04 -> 18.04 upgrade, in text mode, GTK3 and KDE.
Didier Roche-Tolomelli (didrocks) wrote : | # |
Thanks for the review!
There isn't a fixed database structure schema. However, keep in mind that ubuntu-report gather this results (as well as installers one) under some subfields on the json, alongside collecting a lot of others information.
So basically, we have the "To" and a lot of other details. Moreinfo on https:/
For now, we collect those details, similarly than for ubiquity, and we'll organize them next cycle if nobody on IS step up first.
Typos fixed and pushed. Thanks!
- 3132. By Didier Roche-Tolomelli
-
Fix some typos
- 3133. By Didier Roche-Tolomelli
-
Use uptime time instead of time.time(), which is sensitive to system
clock reset by NTP or others… - 3134. By Didier Roche-Tolomelli
-
Fetch if users enabled third party repos before upgrade
- 3135. By Didier Roche-Tolomelli
-
Fetch original install media
Brian Murray (brian-murray) wrote : | # |
This looks good to me to merge and upload after the one change is made.
While this may be more useful for the failure case you might consider including what components (-updates, -security, main, universe, etc...) are enabled on the system that was upgraded. Here's an example from an upgrade log file.
2018-05-12 20:32:36,997 DEBUG found components: {'bionic-updates': {'main', 'universe', 'multiverse', 'restricted'}, 'bionic-security': {'main', 'universe', 'multiverse', 'restricted'}, 'bionic': {'main', 'universe', 'multiverse', 'restricted'}}
Didier Roche-Tolomelli (didrocks) wrote : | # |
We don't want to include the enabled components for now, but thanks for the suggestion! I'll fix the current uptime comment.
Thanks again for the review :)
Brian Murray (brian-murray) : | # |
Didier Roche-Tolomelli (didrocks) wrote : | # |
Tested artful -> bionic upgrade with this code and it reports the expected telemetry data.
Preview Diff
1 | === modified file 'DistUpgrade/DistUpgradeController.py' | |||
2 | --- DistUpgrade/DistUpgradeController.py 2018-03-16 17:56:15 +0000 | |||
3 | +++ DistUpgrade/DistUpgradeController.py 2018-05-18 10:01:29 +0000 | |||
4 | @@ -33,6 +33,7 @@ | |||
5 | 33 | import copy | 33 | import copy |
6 | 34 | from configparser import NoOptionError | 34 | from configparser import NoOptionError |
7 | 35 | from configparser import ConfigParser as SafeConfigParser | 35 | from configparser import ConfigParser as SafeConfigParser |
8 | 36 | from .telemetry import get as get_telemetry | ||
9 | 36 | from .utils import (country_mirror, | 37 | from .utils import (country_mirror, |
10 | 37 | url_downloadable, | 38 | url_downloadable, |
11 | 38 | check_and_fix_xbit, | 39 | check_and_fix_xbit, |
12 | @@ -45,14 +46,7 @@ | |||
13 | 45 | from string import Template | 46 | from string import Template |
14 | 46 | from urllib.parse import urlsplit | 47 | from urllib.parse import urlsplit |
15 | 47 | 48 | ||
24 | 48 | from .DistUpgradeView import ( | 49 | from .DistUpgradeView import Step |
17 | 49 | STEP_PREPARE, | ||
18 | 50 | STEP_MODIFY_SOURCES, | ||
19 | 51 | STEP_FETCH, | ||
20 | 52 | STEP_INSTALL, | ||
21 | 53 | STEP_CLEANUP, | ||
22 | 54 | STEP_REBOOT, | ||
23 | 55 | ) | ||
25 | 56 | from .DistUpgradeCache import MyCache | 50 | from .DistUpgradeCache import MyCache |
26 | 57 | from .DistUpgradeConfigParser import DistUpgradeConfig | 51 | from .DistUpgradeConfigParser import DistUpgradeConfig |
27 | 58 | from .DistUpgradeQuirks import DistUpgradeQuirks | 52 | from .DistUpgradeQuirks import DistUpgradeQuirks |
28 | @@ -866,6 +860,7 @@ | |||
29 | 866 | "'software-properties' tool or " | 860 | "'software-properties' tool or " |
30 | 867 | "your package manager." | 861 | "your package manager." |
31 | 868 | )) | 862 | )) |
32 | 863 | get_telemetry().set_using_third_party_sources(self.sources_disabled) | ||
33 | 869 | return True | 864 | return True |
34 | 870 | 865 | ||
35 | 871 | def _logChanges(self): | 866 | def _logChanges(self): |
36 | @@ -1760,7 +1755,7 @@ | |||
37 | 1760 | def fullUpgrade(self): | 1755 | def fullUpgrade(self): |
38 | 1761 | # sanity check (check for ubuntu-desktop, brokenCache etc) | 1756 | # sanity check (check for ubuntu-desktop, brokenCache etc) |
39 | 1762 | self._view.updateStatus(_("Checking package manager")) | 1757 | self._view.updateStatus(_("Checking package manager")) |
41 | 1763 | self._view.setStep(STEP_PREPARE) | 1758 | self._view.setStep(Step.PREPARE) |
42 | 1764 | 1759 | ||
43 | 1765 | if not self.prepare(): | 1760 | if not self.prepare(): |
44 | 1766 | logging.error("self.prepare() failed") | 1761 | logging.error("self.prepare() failed") |
45 | @@ -1829,7 +1824,7 @@ | |||
46 | 1829 | self.abort() | 1824 | self.abort() |
47 | 1830 | 1825 | ||
48 | 1831 | # update sources.list | 1826 | # update sources.list |
50 | 1832 | self._view.setStep(STEP_MODIFY_SOURCES) | 1827 | self._view.setStep(Step.MODIFY_SOURCES) |
51 | 1833 | self._view.updateStatus(_("Updating repository information")) | 1828 | self._view.updateStatus(_("Updating repository information")) |
52 | 1834 | if not self.updateSourcesList(): | 1829 | if not self.updateSourcesList(): |
53 | 1835 | self.abort() | 1830 | self.abort() |
54 | @@ -1901,14 +1896,14 @@ | |||
55 | 1901 | self.abort() | 1896 | self.abort() |
56 | 1902 | 1897 | ||
57 | 1903 | # fetch the stuff | 1898 | # fetch the stuff |
59 | 1904 | self._view.setStep(STEP_FETCH) | 1899 | self._view.setStep(Step.FETCH) |
60 | 1905 | self._view.updateStatus(_("Fetching")) | 1900 | self._view.updateStatus(_("Fetching")) |
61 | 1906 | if not self.doDistUpgradeFetching(): | 1901 | if not self.doDistUpgradeFetching(): |
62 | 1907 | self._enableAptCronJob() | 1902 | self._enableAptCronJob() |
63 | 1908 | self.abort() | 1903 | self.abort() |
64 | 1909 | 1904 | ||
65 | 1910 | # now do the upgrade | 1905 | # now do the upgrade |
67 | 1911 | self._view.setStep(STEP_INSTALL) | 1906 | self._view.setStep(Step.INSTALL) |
68 | 1912 | self._view.updateStatus(_("Upgrading")) | 1907 | self._view.updateStatus(_("Upgrading")) |
69 | 1913 | if not self.doDistUpgrade(): | 1908 | if not self.doDistUpgrade(): |
70 | 1914 | # run the post install scripts (for stuff like UUID conversion) | 1909 | # run the post install scripts (for stuff like UUID conversion) |
71 | @@ -1922,7 +1917,7 @@ | |||
72 | 1922 | sys.exit(1) | 1917 | sys.exit(1) |
73 | 1923 | 1918 | ||
74 | 1924 | # do post-upgrade stuff | 1919 | # do post-upgrade stuff |
76 | 1925 | self._view.setStep(STEP_CLEANUP) | 1920 | self._view.setStep(Step.CLEANUP) |
77 | 1926 | self._view.updateStatus(_("Searching for obsolete software")) | 1921 | self._view.updateStatus(_("Searching for obsolete software")) |
78 | 1927 | self.doPostUpgrade() | 1922 | self.doPostUpgrade() |
79 | 1928 | 1923 | ||
80 | @@ -1935,8 +1930,9 @@ | |||
81 | 1935 | os.unlink("/var/lib/ubuntu-release-upgrader/release-upgrade-available") | 1930 | os.unlink("/var/lib/ubuntu-release-upgrader/release-upgrade-available") |
82 | 1936 | 1931 | ||
83 | 1937 | # done, ask for reboot | 1932 | # done, ask for reboot |
85 | 1938 | self._view.setStep(STEP_REBOOT) | 1933 | self._view.setStep(Step.REBOOT) |
86 | 1939 | self._view.updateStatus(_("System upgrade is complete.")) | 1934 | self._view.updateStatus(_("System upgrade is complete.")) |
87 | 1935 | get_telemetry().done() | ||
88 | 1940 | # FIXME should we look into /var/run/reboot-required here? | 1936 | # FIXME should we look into /var/run/reboot-required here? |
89 | 1941 | if (not inside_chroot() and | 1937 | if (not inside_chroot() and |
90 | 1942 | self._view.confirmRestart()): | 1938 | self._view.confirmRestart()): |
91 | @@ -1950,20 +1946,20 @@ | |||
92 | 1950 | 1946 | ||
93 | 1951 | def doPartialUpgrade(self): | 1947 | def doPartialUpgrade(self): |
94 | 1952 | " partial upgrade mode, useful for repairing " | 1948 | " partial upgrade mode, useful for repairing " |
98 | 1953 | self._view.setStep(STEP_PREPARE) | 1949 | self._view.setStep(Step.PREPARE) |
99 | 1954 | self._view.hideStep(STEP_MODIFY_SOURCES) | 1950 | self._view.hideStep(Step.MODIFY_SOURCES) |
100 | 1955 | self._view.hideStep(STEP_REBOOT) | 1951 | self._view.hideStep(Step.REBOOT) |
101 | 1956 | self._partialUpgrade = True | 1952 | self._partialUpgrade = True |
102 | 1957 | self.prepare() | 1953 | self.prepare() |
103 | 1958 | if not self.doPostInitialUpdate(): | 1954 | if not self.doPostInitialUpdate(): |
104 | 1959 | return False | 1955 | return False |
105 | 1960 | if not self.askDistUpgrade(): | 1956 | if not self.askDistUpgrade(): |
106 | 1961 | return False | 1957 | return False |
108 | 1962 | self._view.setStep(STEP_FETCH) | 1958 | self._view.setStep(Step.FETCH) |
109 | 1963 | self._view.updateStatus(_("Fetching")) | 1959 | self._view.updateStatus(_("Fetching")) |
110 | 1964 | if not self.doDistUpgradeFetching(): | 1960 | if not self.doDistUpgradeFetching(): |
111 | 1965 | return False | 1961 | return False |
113 | 1966 | self._view.setStep(STEP_INSTALL) | 1962 | self._view.setStep(Step.INSTALL) |
114 | 1967 | self._view.updateStatus(_("Upgrading")) | 1963 | self._view.updateStatus(_("Upgrading")) |
115 | 1968 | if not self.doDistUpgrade(): | 1964 | if not self.doDistUpgrade(): |
116 | 1969 | self._view.information(_("Upgrade complete"), | 1965 | self._view.information(_("Upgrade complete"), |
117 | @@ -1971,7 +1967,7 @@ | |||
118 | 1971 | "were errors during the upgrade " | 1967 | "were errors during the upgrade " |
119 | 1972 | "process.")) | 1968 | "process.")) |
120 | 1973 | return False | 1969 | return False |
122 | 1974 | self._view.setStep(STEP_CLEANUP) | 1970 | self._view.setStep(Step.CLEANUP) |
123 | 1975 | if not self.doPostUpgrade(): | 1971 | if not self.doPostUpgrade(): |
124 | 1976 | self._view.information(_("Upgrade complete"), | 1972 | self._view.information(_("Upgrade complete"), |
125 | 1977 | _("The upgrade has completed but there " | 1973 | _("The upgrade has completed but there " |
126 | 1978 | 1974 | ||
127 | === modified file 'DistUpgrade/DistUpgradeView.py' | |||
128 | --- DistUpgrade/DistUpgradeView.py 2017-05-18 17:33:48 +0000 | |||
129 | +++ DistUpgrade/DistUpgradeView.py 2018-05-18 10:01:29 +0000 | |||
130 | @@ -21,7 +21,9 @@ | |||
131 | 21 | 21 | ||
132 | 22 | from .DistUpgradeGettext import gettext as _ | 22 | from .DistUpgradeGettext import gettext as _ |
133 | 23 | from .DistUpgradeGettext import ngettext | 23 | from .DistUpgradeGettext import ngettext |
134 | 24 | from .telemetry import get as get_telemetry | ||
135 | 24 | import apt | 25 | import apt |
136 | 26 | from enum import Enum | ||
137 | 25 | import errno | 27 | import errno |
138 | 26 | import os | 28 | import os |
139 | 27 | import apt_pkg | 29 | import apt_pkg |
140 | @@ -252,13 +254,14 @@ | |||
141 | 252 | def hide(self): | 254 | def hide(self): |
142 | 253 | pass | 255 | pass |
143 | 254 | 256 | ||
151 | 255 | (STEP_PREPARE, | 257 | class Step(Enum): |
152 | 256 | STEP_MODIFY_SOURCES, | 258 | PREPARE = 1 |
153 | 257 | STEP_FETCH, | 259 | MODIFY_SOURCES = 2 |
154 | 258 | STEP_INSTALL, | 260 | FETCH = 3 |
155 | 259 | STEP_CLEANUP, | 261 | INSTALL = 4 |
156 | 260 | STEP_REBOOT, | 262 | CLEANUP = 5 |
157 | 261 | STEP_N) = range(1,8) | 263 | REBOOT = 6 |
158 | 264 | N = 7 | ||
159 | 262 | 265 | ||
160 | 263 | # Declare these translatable strings from the .ui files here so that | 266 | # Declare these translatable strings from the .ui files here so that |
161 | 264 | # xgettext picks them up. | 267 | # xgettext picks them up. |
162 | @@ -304,6 +307,7 @@ | |||
163 | 304 | 4. Post upgrade stuff | 307 | 4. Post upgrade stuff |
164 | 305 | 5. Complete | 308 | 5. Complete |
165 | 306 | """ | 309 | """ |
166 | 310 | get_telemetry().add_stage(step.name) | ||
167 | 307 | pass | 311 | pass |
168 | 308 | def hideStep(self, step): | 312 | def hideStep(self, step): |
169 | 309 | " hide a certain step from the GUI " | 313 | " hide a certain step from the GUI " |
170 | 310 | 314 | ||
171 | === modified file 'DistUpgrade/DistUpgradeViewGtk3.py' | |||
172 | --- DistUpgrade/DistUpgradeViewGtk3.py 2016-02-23 17:06:42 +0000 | |||
173 | +++ DistUpgrade/DistUpgradeViewGtk3.py 2018-05-18 10:01:29 +0000 | |||
174 | @@ -52,6 +52,7 @@ | |||
175 | 52 | from .DistUpgradeApport import run_apport, apport_crash | 52 | from .DistUpgradeApport import run_apport, apport_crash |
176 | 53 | 53 | ||
177 | 54 | from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress | 54 | from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress |
178 | 55 | from .telemetry import get as get_telemetry | ||
179 | 55 | from .SimpleGtk3builderApp import SimpleGtkbuilderApp | 56 | from .SimpleGtk3builderApp import SimpleGtkbuilderApp |
180 | 56 | 57 | ||
181 | 57 | import gettext | 58 | import gettext |
182 | @@ -70,7 +71,7 @@ | |||
183 | 70 | """ update is called regularly so that the gui can be redrawn """ | 71 | """ update is called regularly so that the gui can be redrawn """ |
184 | 71 | if text: | 72 | if text: |
185 | 72 | self.status.set_text(text) | 73 | self.status.set_text(text) |
187 | 73 | self.progress.set_fraction(step/float(self.totalSteps)) | 74 | self.progress.set_fraction(step.value/float(self.totalSteps)) |
188 | 74 | while Gtk.events_pending(): | 75 | while Gtk.events_pending(): |
189 | 75 | Gtk.main_iteration() | 76 | Gtk.main_iteration() |
190 | 76 | def ask_cdrom_name(self): | 77 | def ask_cdrom_name(self): |
191 | @@ -452,6 +453,8 @@ | |||
192 | 452 | # check if we have a display etc | 453 | # check if we have a display etc |
193 | 453 | Gtk.init_check(sys.argv) | 454 | Gtk.init_check(sys.argv) |
194 | 454 | 455 | ||
195 | 456 | get_telemetry().set_updater_type('GTK') | ||
196 | 457 | |||
197 | 455 | try: | 458 | try: |
198 | 456 | locale.bindtextdomain("ubuntu-release-upgrader",localedir) | 459 | locale.bindtextdomain("ubuntu-release-upgrader",localedir) |
199 | 457 | gettext.textdomain("ubuntu-release-upgrader") | 460 | gettext.textdomain("ubuntu-release-upgrader") |
200 | @@ -472,7 +475,7 @@ | |||
201 | 472 | # terminal stuff | 475 | # terminal stuff |
202 | 473 | self.create_terminal() | 476 | self.create_terminal() |
203 | 474 | 477 | ||
205 | 475 | self.prev_step = 0 # keep a record of the latest step | 478 | self.prev_step = None # keep a record of the latest step |
206 | 476 | # we don't use this currently | 479 | # we don't use this currently |
207 | 477 | #self.window_main.set_keep_above(True) | 480 | #self.window_main.set_keep_above(True) |
208 | 478 | self.icontheme = Gtk.IconTheme.get_default() | 481 | self.icontheme = Gtk.IconTheme.get_default() |
209 | @@ -598,44 +601,45 @@ | |||
210 | 598 | def updateStatus(self, msg): | 601 | def updateStatus(self, msg): |
211 | 599 | self.label_status.set_text("%s" % msg) | 602 | self.label_status.set_text("%s" % msg) |
212 | 600 | def hideStep(self, step): | 603 | def hideStep(self, step): |
216 | 601 | image = getattr(self,"image_step%i" % step) | 604 | image = getattr(self,"image_step%i" % step.value) |
217 | 602 | label = getattr(self,"label_step%i" % step) | 605 | label = getattr(self,"label_step%i" % step.value) |
218 | 603 | #arrow = getattr(self,"arrow_step%i" % step) | 606 | #arrow = getattr(self,"arrow_step%i" % step.value) |
219 | 604 | image.hide() | 607 | image.hide() |
220 | 605 | label.hide() | 608 | label.hide() |
221 | 606 | def showStep(self, step): | 609 | def showStep(self, step): |
224 | 607 | image = getattr(self,"image_step%i" % step) | 610 | image = getattr(self,"image_step%i" % step.value) |
225 | 608 | label = getattr(self,"label_step%i" % step) | 611 | label = getattr(self,"label_step%i" % step.value) |
226 | 609 | image.show() | 612 | image.show() |
227 | 610 | label.show() | 613 | label.show() |
228 | 611 | def abort(self): | 614 | def abort(self): |
229 | 612 | size = Gtk.IconSize.MENU | 615 | size = Gtk.IconSize.MENU |
230 | 613 | step = self.prev_step | 616 | step = self.prev_step |
234 | 614 | if step > 0: | 617 | if step: |
235 | 615 | image = getattr(self,"image_step%i" % step) | 618 | image = getattr(self,"image_step%i" % step.value) |
236 | 616 | arrow = getattr(self,"arrow_step%i" % step) | 619 | arrow = getattr(self,"arrow_step%i" % step.value) |
237 | 617 | image.set_from_stock(Gtk.STOCK_CANCEL, size) | 620 | image.set_from_stock(Gtk.STOCK_CANCEL, size) |
238 | 618 | image.show() | 621 | image.show() |
239 | 619 | arrow.hide() | 622 | arrow.hide() |
240 | 620 | def setStep(self, step): | 623 | def setStep(self, step): |
241 | 624 | super(DistUpgradeViewGtk3, self).setStep(step) | ||
242 | 621 | if self.icontheme.rescan_if_needed(): | 625 | if self.icontheme.rescan_if_needed(): |
243 | 622 | logging.debug("icon theme changed, re-reading") | 626 | logging.debug("icon theme changed, re-reading") |
244 | 623 | # first update the "previous" step as completed | 627 | # first update the "previous" step as completed |
245 | 624 | size = Gtk.IconSize.MENU | 628 | size = Gtk.IconSize.MENU |
246 | 625 | attrlist=Pango.AttrList() | 629 | attrlist=Pango.AttrList() |
247 | 626 | if self.prev_step: | 630 | if self.prev_step: |
251 | 627 | image = getattr(self,"image_step%i" % self.prev_step) | 631 | image = getattr(self,"image_step%i" % self.prev_step.value) |
252 | 628 | label = getattr(self,"label_step%i" % self.prev_step) | 632 | label = getattr(self,"label_step%i" % self.prev_step.value) |
253 | 629 | arrow = getattr(self,"arrow_step%i" % self.prev_step) | 633 | arrow = getattr(self,"arrow_step%i" % self.prev_step.value) |
254 | 630 | label.set_property("attributes",attrlist) | 634 | label.set_property("attributes",attrlist) |
255 | 631 | image.set_from_stock(Gtk.STOCK_APPLY, size) | 635 | image.set_from_stock(Gtk.STOCK_APPLY, size) |
256 | 632 | image.show() | 636 | image.show() |
257 | 633 | arrow.hide() | 637 | arrow.hide() |
258 | 634 | self.prev_step = step | 638 | self.prev_step = step |
259 | 635 | # show the an arrow for the current step and make the label bold | 639 | # show the an arrow for the current step and make the label bold |
263 | 636 | image = getattr(self,"image_step%i" % step) | 640 | image = getattr(self,"image_step%i" % step.value) |
264 | 637 | label = getattr(self,"label_step%i" % step) | 641 | label = getattr(self,"label_step%i" % step.value) |
265 | 638 | arrow = getattr(self,"arrow_step%i" % step) | 642 | arrow = getattr(self,"arrow_step%i" % step.value) |
266 | 639 | # check if that step was not hidden with hideStep() | 643 | # check if that step was not hidden with hideStep() |
267 | 640 | if not label.get_property("visible"): | 644 | if not label.get_property("visible"): |
268 | 641 | return | 645 | return |
269 | 642 | 646 | ||
270 | === modified file 'DistUpgrade/DistUpgradeViewKDE.py' | |||
271 | --- DistUpgrade/DistUpgradeViewKDE.py 2018-01-22 11:12:39 +0000 | |||
272 | +++ DistUpgrade/DistUpgradeViewKDE.py 2018-05-18 10:01:29 +0000 | |||
273 | @@ -62,6 +62,7 @@ | |||
274 | 62 | from .DistUpgradeApport import run_apport, apport_crash | 62 | from .DistUpgradeApport import run_apport, apport_crash |
275 | 63 | 63 | ||
276 | 64 | from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress | 64 | from .DistUpgradeView import DistUpgradeView, FuzzyTimeToStr, InstallProgress, AcquireProgress |
277 | 65 | from .telemetry import get as get_telemetry | ||
278 | 65 | 66 | ||
279 | 66 | import select | 67 | import select |
280 | 67 | import gettext | 68 | import gettext |
281 | @@ -251,7 +252,7 @@ | |||
282 | 251 | """ update is called regularly so that the gui can be redrawn """ | 252 | """ update is called regularly so that the gui can be redrawn """ |
283 | 252 | if text: | 253 | if text: |
284 | 253 | self.status.setText(text) | 254 | self.status.setText(text) |
286 | 254 | self.progressbar.setValue(step/float(self.totalSteps)) | 255 | self.progressbar.setValue(step.value/float(self.totalSteps)) |
287 | 255 | QApplication.processEvents() | 256 | QApplication.processEvents() |
288 | 256 | 257 | ||
289 | 257 | def ask_cdrom_name(self): | 258 | def ask_cdrom_name(self): |
290 | @@ -560,6 +561,8 @@ | |||
291 | 560 | """KDE frontend of the distUpgrade tool""" | 561 | """KDE frontend of the distUpgrade tool""" |
292 | 561 | def __init__(self, datadir=None, logdir=None): | 562 | def __init__(self, datadir=None, logdir=None): |
293 | 562 | DistUpgradeView.__init__(self) | 563 | DistUpgradeView.__init__(self) |
294 | 564 | |||
295 | 565 | get_telemetry().set_updater_type('KDE') | ||
296 | 563 | # silence the PyQt4 logger | 566 | # silence the PyQt4 logger |
297 | 564 | logger = logging.getLogger("PyQt4") | 567 | logger = logging.getLogger("PyQt4") |
298 | 565 | logger.setLevel(logging.INFO) | 568 | logger.setLevel(logging.INFO) |
299 | @@ -613,7 +616,7 @@ | |||
300 | 613 | self.window_main.setParent(self) | 616 | self.window_main.setParent(self) |
301 | 614 | self.window_main.show() | 617 | self.window_main.show() |
302 | 615 | 618 | ||
304 | 616 | self.prev_step = 0 # keep a record of the latest step | 619 | self.prev_step = None # keep a record of the latest step |
305 | 617 | 620 | ||
306 | 618 | self._opCacheProgress = KDEOpProgress(self.window_main.progressbar_cache, self.window_main.progress_text) | 621 | self._opCacheProgress = KDEOpProgress(self.window_main.progressbar_cache, self.window_main.progress_text) |
307 | 619 | self._acquireProgress = KDEAcquireProgressAdapter(self) | 622 | self._acquireProgress = KDEAcquireProgressAdapter(self) |
308 | @@ -733,15 +736,15 @@ | |||
309 | 733 | self.window_main.label_status.setText(msg) | 736 | self.window_main.label_status.setText(msg) |
310 | 734 | 737 | ||
311 | 735 | def hideStep(self, step): | 738 | def hideStep(self, step): |
314 | 736 | image = getattr(self.window_main,"image_step%i" % step) | 739 | image = getattr(self.window_main,"image_step%i" % step.value) |
315 | 737 | label = getattr(self.window_main,"label_step%i" % step) | 740 | label = getattr(self.window_main,"label_step%i" % step.value) |
316 | 738 | image.hide() | 741 | image.hide() |
317 | 739 | label.hide() | 742 | label.hide() |
318 | 740 | 743 | ||
319 | 741 | def abort(self): | 744 | def abort(self): |
320 | 742 | step = self.prev_step | 745 | step = self.prev_step |
323 | 743 | if step > 0: | 746 | if step: |
324 | 744 | image = getattr(self.window_main,"image_step%i" % step) | 747 | image = getattr(self.window_main,"image_step%i" % step.value) |
325 | 745 | cancelIcon = _icon("dialog-cancel", | 748 | cancelIcon = _icon("dialog-cancel", |
326 | 746 | fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-cancel.png", | 749 | fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-cancel.png", |
327 | 747 | "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-cancel.png", | 750 | "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-cancel.png", |
328 | @@ -750,6 +753,7 @@ | |||
329 | 750 | image.show() | 753 | image.show() |
330 | 751 | 754 | ||
331 | 752 | def setStep(self, step): | 755 | def setStep(self, step): |
332 | 756 | super(DistUpgradeViewKDE , self).setStep(step) | ||
333 | 753 | okIcon = _icon("dialog-ok", | 757 | okIcon = _icon("dialog-ok", |
334 | 754 | fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-ok.png", | 758 | fallbacks=["/usr/share/icons/oxygen/16x16/actions/dialog-ok.png", |
335 | 755 | "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-ok.png", | 759 | "/usr/lib/kde4/share/icons/oxygen/16x16/actions/dialog-ok.png", |
336 | @@ -760,15 +764,15 @@ | |||
337 | 760 | "/usr/share/icons/crystalsvg/16x16/actions/1rightarrow.png"]) | 764 | "/usr/share/icons/crystalsvg/16x16/actions/1rightarrow.png"]) |
338 | 761 | 765 | ||
339 | 762 | if self.prev_step: | 766 | if self.prev_step: |
342 | 763 | image = getattr(self.window_main,"image_step%i" % self.prev_step) | 767 | image = getattr(self.window_main,"image_step%i" % self.prev_step.value) |
343 | 764 | label = getattr(self.window_main,"label_step%i" % self.prev_step) | 768 | label = getattr(self.window_main,"label_step%i" % self.prev_step.value) |
344 | 765 | image.setPixmap(okIcon.pixmap(16, 16)) | 769 | image.setPixmap(okIcon.pixmap(16, 16)) |
345 | 766 | image.show() | 770 | image.show() |
346 | 767 | ##arrow.hide() | 771 | ##arrow.hide() |
347 | 768 | self.prev_step = step | 772 | self.prev_step = step |
348 | 769 | # show the an arrow for the current step and make the label bold | 773 | # show the an arrow for the current step and make the label bold |
351 | 770 | image = getattr(self.window_main,"image_step%i" % step) | 774 | image = getattr(self.window_main,"image_step%i" % step.value) |
352 | 771 | label = getattr(self.window_main,"label_step%i" % step) | 775 | label = getattr(self.window_main,"label_step%i" % step.value) |
353 | 772 | image.setPixmap(arrowIcon.pixmap(16, 16)) | 776 | image.setPixmap(arrowIcon.pixmap(16, 16)) |
354 | 773 | image.show() | 777 | image.show() |
355 | 774 | label.setText("<b>" + label.text() + "</b>") | 778 | label.setText("<b>" + label.text() + "</b>") |
356 | 775 | 779 | ||
357 | === modified file 'DistUpgrade/DistUpgradeViewNonInteractive.py' | |||
358 | --- DistUpgrade/DistUpgradeViewNonInteractive.py 2016-10-10 18:05:08 +0000 | |||
359 | +++ DistUpgrade/DistUpgradeViewNonInteractive.py 2018-05-18 10:01:29 +0000 | |||
360 | @@ -36,6 +36,7 @@ | |||
361 | 36 | from subprocess import PIPE, Popen | 36 | from subprocess import PIPE, Popen |
362 | 37 | 37 | ||
363 | 38 | from .DistUpgradeView import DistUpgradeView, InstallProgress, AcquireProgress | 38 | from .DistUpgradeView import DistUpgradeView, InstallProgress, AcquireProgress |
364 | 39 | from .telemetry import get as get_telemetry | ||
365 | 39 | from .DistUpgradeConfigParser import DistUpgradeConfig | 40 | from .DistUpgradeConfigParser import DistUpgradeConfig |
366 | 40 | 41 | ||
367 | 41 | 42 | ||
368 | @@ -251,6 +252,7 @@ | |||
369 | 251 | " non-interactive version of the upgrade view " | 252 | " non-interactive version of the upgrade view " |
370 | 252 | def __init__(self, datadir=None, logdir=None): | 253 | def __init__(self, datadir=None, logdir=None): |
371 | 253 | DistUpgradeView.__init__(self) | 254 | DistUpgradeView.__init__(self) |
372 | 255 | get_telemetry().set_updater_type('NonInteractive') | ||
373 | 254 | self.config = DistUpgradeConfig(".") | 256 | self.config = DistUpgradeConfig(".") |
374 | 255 | self._acquireProgress = NonInteractiveAcquireProgress() | 257 | self._acquireProgress = NonInteractiveAcquireProgress() |
375 | 256 | self._installProgress = NonInteractiveInstallProgress(logdir) | 258 | self._installProgress = NonInteractiveInstallProgress(logdir) |
376 | @@ -286,6 +288,7 @@ | |||
377 | 286 | 4. Post upgrade stuff | 288 | 4. Post upgrade stuff |
378 | 287 | 5. Complete | 289 | 5. Complete |
379 | 288 | """ | 290 | """ |
380 | 291 | super(DistUpgradeViewNonInteractive, self).setStep(step) | ||
381 | 289 | pass | 292 | pass |
382 | 290 | def confirmChanges(self, summary, changes, demotions, downloadSize, | 293 | def confirmChanges(self, summary, changes, demotions, downloadSize, |
383 | 291 | actions=None, removal_bold=True): | 294 | actions=None, removal_bold=True): |
384 | 292 | 295 | ||
385 | === modified file 'DistUpgrade/DistUpgradeViewText.py' | |||
386 | --- DistUpgrade/DistUpgradeViewText.py 2016-04-22 19:33:04 +0000 | |||
387 | +++ DistUpgrade/DistUpgradeViewText.py 2018-05-18 10:01:29 +0000 | |||
388 | @@ -36,6 +36,7 @@ | |||
389 | 36 | ENCODING, | 36 | ENCODING, |
390 | 37 | InstallProgress, | 37 | InstallProgress, |
391 | 38 | ) | 38 | ) |
392 | 39 | from .telemetry import get as get_telemetry | ||
393 | 39 | import apt.progress | 40 | import apt.progress |
394 | 40 | 41 | ||
395 | 41 | import gettext | 42 | import gettext |
396 | @@ -88,7 +89,7 @@ | |||
397 | 88 | def update(self, text, step): | 89 | def update(self, text, step): |
398 | 89 | """ update is called regularly so that the gui can be redrawn """ | 90 | """ update is called regularly so that the gui can be redrawn """ |
399 | 90 | if text: | 91 | if text: |
401 | 91 | print("%s (%f)" % (text, step/float(self.totalSteps)*100)) | 92 | print("%s (%f)" % (text, step.value/float(self.totalSteps)*100)) |
402 | 92 | def ask_cdrom_name(self): | 93 | def ask_cdrom_name(self): |
403 | 93 | return (False, "") | 94 | return (False, "") |
404 | 94 | def change_cdrom(self): | 95 | def change_cdrom(self): |
405 | @@ -101,6 +102,7 @@ | |||
406 | 101 | def __init__(self, datadir=None, logdir=None): | 102 | def __init__(self, datadir=None, logdir=None): |
407 | 102 | # indicate that we benefit from using gnu screen | 103 | # indicate that we benefit from using gnu screen |
408 | 103 | self.needs_screen = True | 104 | self.needs_screen = True |
409 | 105 | get_telemetry().set_updater_type('Text') | ||
410 | 104 | # its important to have a debconf frontend for | 106 | # its important to have a debconf frontend for |
411 | 105 | # packages like "quagga" | 107 | # packages like "quagga" |
412 | 106 | if "DEBIAN_FRONTEND" not in os.environ: | 108 | if "DEBIAN_FRONTEND" not in os.environ: |
413 | @@ -116,7 +118,7 @@ | |||
414 | 116 | except Exception as e: | 118 | except Exception as e: |
415 | 117 | logging.warning("Error setting locales (%s)" % e) | 119 | logging.warning("Error setting locales (%s)" % e) |
416 | 118 | 120 | ||
418 | 119 | self.last_step = 0 # keep a record of the latest step | 121 | self.last_step = None # keep a record of the latest step |
419 | 120 | self._opCacheProgress = apt.progress.text.OpProgress() | 122 | self._opCacheProgress = apt.progress.text.OpProgress() |
420 | 121 | self._acquireProgress = TextAcquireProgress() | 123 | self._acquireProgress = TextAcquireProgress() |
421 | 122 | self._cdromProgress = TextCdromProgressAdapter() | 124 | self._cdromProgress = TextCdromProgressAdapter() |
422 | @@ -162,6 +164,7 @@ | |||
423 | 162 | print() | 164 | print() |
424 | 163 | print(_("Aborting")) | 165 | print(_("Aborting")) |
425 | 164 | def setStep(self, step): | 166 | def setStep(self, step): |
426 | 167 | super(DistUpgradeViewText, self).setStep(step) | ||
427 | 165 | self.last_step = step | 168 | self.last_step = step |
428 | 166 | def showDemotions(self, summary, msg, demotions): | 169 | def showDemotions(self, summary, msg, demotions): |
429 | 167 | self.information(summary, msg, | 170 | self.information(summary, msg, |
430 | 168 | 171 | ||
431 | === added file 'DistUpgrade/telemetry.py' | |||
432 | --- DistUpgrade/telemetry.py 1970-01-01 00:00:00 +0000 | |||
433 | +++ DistUpgrade/telemetry.py 2018-05-18 10:01:29 +0000 | |||
434 | @@ -0,0 +1,101 @@ | |||
435 | 1 | # -*- coding: utf-8; Mode: Python; indent-tabs-mode: nil; tab-width: 4 -*- | ||
436 | 2 | |||
437 | 3 | # Copyright (C) 2018 Canonical Ltd. | ||
438 | 4 | # | ||
439 | 5 | # Functions useful for the final install.py script and for ubiquity | ||
440 | 6 | # plugins to use | ||
441 | 7 | # | ||
442 | 8 | # This program is free software; you can redistribute it and/or modify | ||
443 | 9 | # it under the terms of the GNU General Public License as published by | ||
444 | 10 | # the Free Software Foundation; either version 2 of the License, or | ||
445 | 11 | # (at your option) any later version. | ||
446 | 12 | # | ||
447 | 13 | # This program is distributed in the hope that it will be useful, | ||
448 | 14 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
449 | 15 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
450 | 16 | # GNU General Public License for more details. | ||
451 | 17 | # | ||
452 | 18 | # You should have received a copy of the GNU General Public License | ||
453 | 19 | # along with this program; if not, write to the Free Software | ||
454 | 20 | # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA | ||
455 | 21 | |||
456 | 22 | |||
457 | 23 | import logging | ||
458 | 24 | import json | ||
459 | 25 | import os | ||
460 | 26 | import stat | ||
461 | 27 | import subprocess | ||
462 | 28 | |||
463 | 29 | |||
464 | 30 | def get(): | ||
465 | 31 | """Return a singleton _Telemetry instance.""" | ||
466 | 32 | if _Telemetry._telemetry is None: | ||
467 | 33 | _Telemetry._telemetry = _Telemetry() | ||
468 | 34 | return _Telemetry._telemetry | ||
469 | 35 | |||
470 | 36 | |||
471 | 37 | class _Telemetry(): | ||
472 | 38 | |||
473 | 39 | _telemetry = None | ||
474 | 40 | |||
475 | 41 | def __init__(self): | ||
476 | 42 | self._metrics = {} | ||
477 | 43 | self._stages_hist = {} | ||
478 | 44 | self._start_time = self._get_current_uptime() | ||
479 | 45 | self._metrics["From"] = subprocess.Popen( | ||
480 | 46 | ["lsb_release", "-r", "-s"], stdout=subprocess.PIPE, | ||
481 | 47 | universal_newlines=True).communicate()[0].strip() | ||
482 | 48 | self.add_stage('start') | ||
483 | 49 | self._dest_path = '/var/log/upgrade/telemetry' | ||
484 | 50 | try: | ||
485 | 51 | with open('/var/log/installer/media-info') as f: | ||
486 | 52 | self._metrics['InstallMedia'] = f.readline() | ||
487 | 53 | except FileNotFoundError: | ||
488 | 54 | pass | ||
489 | 55 | |||
490 | 56 | def _get_current_uptime(self): | ||
491 | 57 | """Get current uptime info. None if we couldn't fetch it.""" | ||
492 | 58 | uptime = None | ||
493 | 59 | try: | ||
494 | 60 | with open('/proc/uptime') as f: | ||
495 | 61 | uptime = float(f.read().split()[0]) | ||
496 | 62 | except (FileNotFoundError, OSError, ValueError) as e: | ||
497 | 63 | logging.warning("Exception while fetching current uptime: " + str(e)) | ||
498 | 64 | return uptime | ||
499 | 65 | |||
500 | 66 | def add_stage(self, stage_name): | ||
501 | 67 | """Record installer stage with current time""" | ||
502 | 68 | now = self._get_current_uptime() | ||
503 | 69 | if self._start_time is None or now is None: | ||
504 | 70 | return | ||
505 | 71 | self._stages_hist[int(now-self._start_time)] = stage_name | ||
506 | 72 | |||
507 | 73 | def set_updater_type(self, updater_type): | ||
508 | 74 | """Record updater type""" | ||
509 | 75 | self._metrics['Type'] = updater_type | ||
510 | 76 | |||
511 | 77 | def set_using_third_party_sources(self, using): | ||
512 | 78 | """Record if the user had third party sources""" | ||
513 | 79 | self._metrics['ThirdPartySources'] = using | ||
514 | 80 | |||
515 | 81 | def done(self): | ||
516 | 82 | """Close telemetry collection | ||
517 | 83 | |||
518 | 84 | Save to destination file""" | ||
519 | 85 | |||
520 | 86 | self._metrics['Stages'] = self._stages_hist | ||
521 | 87 | |||
522 | 88 | target_dir = os.path.dirname(self._dest_path) | ||
523 | 89 | try: | ||
524 | 90 | if not os.path.exists(target_dir): | ||
525 | 91 | os.makedirs(target_dir) | ||
526 | 92 | with open(self._dest_path, 'w') as f: | ||
527 | 93 | json.dump(self._metrics, f) | ||
528 | 94 | os.chmod(self._dest_path, | ||
529 | 95 | stat.S_IRUSR | stat.S_IWUSR | | ||
530 | 96 | stat.S_IRGRP | stat.S_IROTH) | ||
531 | 97 | except OSError as e: | ||
532 | 98 | logging.warning("Exception while storing telemetry data: " + | ||
533 | 99 | str(e)) | ||
534 | 100 | |||
535 | 101 | # vim:ai:et:sts=4:tw=80:sw=4: | ||
536 | 0 | 102 | ||
537 | === modified file 'debian/changelog' | |||
538 | --- debian/changelog 2018-05-02 19:43:50 +0000 | |||
539 | +++ debian/changelog 2018-05-18 10:01:29 +0000 | |||
540 | @@ -1,3 +1,9 @@ | |||
541 | 1 | ubuntu-release-upgrader (1:18.04.19) UNRELEASED; urgency=medium | ||
542 | 2 | |||
543 | 3 | * Add upgrade telemetry information (LP: #1755456) | ||
544 | 4 | |||
545 | 5 | -- Didier Roche <didrocks@ubuntu.com> Thu, 03 May 2018 15:55:34 +0200 | ||
546 | 6 | |||
547 | 1 | ubuntu-release-upgrader (1:18.04.18) bionic; urgency=medium | 7 | ubuntu-release-upgrader (1:18.04.18) bionic; urgency=medium |
548 | 2 | 8 | ||
549 | 3 | * data/removal_blacklist.cfg: Drop unity, ubuntu-gnome-desktop, and | 9 | * data/removal_blacklist.cfg: Drop unity, ubuntu-gnome-desktop, and |
Keep in mind that it is possible to from one release to multiple releases e.g. from xenial to artful or bionic. So without knowing the database structure I think it would make sense to also record the "To" release. You can find this in DistUpgradeCont roller. py.
I'm also curious about how this information will be used because the use cases might benefit from gathering additional information like the version of release upgrader being used or environmental variables set.
Otherwise there are a couple of typos and one nitpick to have a look at.