Distribution upgrade to 18.04: used nvidia display driver is commented out

Bug #1773637 reported by Yoshee
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
nvidia-graphics-drivers-390 (Ubuntu)
Won't Fix
Undecided
Unassigned
Cosmic
Won't Fix
Undecided
Unassigned
ubuntu-release-upgrader (Ubuntu)
Fix Released
High
Brian Murray
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Fix Released
Undecided
Unassigned

Bug Description

Impact
------
Users who have manually configured an xorg.conf file with nvidia enabled as a driver will be surprised to have disabled on upgrade to a new release of Ubuntu and won't have a graphical display.

Test Case
---------
If you don't have a system with nvidia hardware to upgrade this is a bit convoluted but still works.

1) Run do-release-upgrade (use -d if upgrading to disco)
2) Cancel the upgrade
3) Create an /etc/X11/xorg.conf file with a Device section listed in the original description using the nvidia driver.
4) cd to /tmp/ubuntu-release-upgrader-$TEMPNAME
5) run ./$target-release-name --frontend DistUpgradeViewText
6) Observe the Driver line is commented out of xorg.conf

When testing the release-upgrader from -proposed you'll need to use the -p switch to get the right tarball. Additionally, you'll want to add a step 4.5 where you modify xorg_fix_proprietary.py so nvidia defaults to True instead of False (line 106) this'll simulate the nvidia driver being loaded.

Regression Potential
--------------------
This just changes how xorg_fix_proprietary check for nvidia being in use and it only affects people who have an /etc/X11/xorg.conf so the impact of a regression is rather small. Additionally, the logic used to determine if nvidia in use is the same logic used in the apport package hook for all xorg packages so is well tested.

Original Description
--------------------
I upgraded my system from 17.10 to 18.04 Bionic Beaver via do-release-upgrade.
After the installation had finished, Xorg wouldn't start. I later found out that this was because the installation routine had commented out the nvidia display driver I used in the xorg.conf:

Section "Device"
    Identifier "nvidia"
# Driver "nvidia"
    BusID "PCI:10@0:0:0"
    Option "ConstrainCursor" "off"
EndSection

As the installtion routine backuped my xorg.conf I could verify that the line hadn't been commented out before upgrading. Removing the "#" solved the problem.

ProblemType: BugDistroRelease: Ubuntu 18.04
Package: xorg 1:7.7+19ubuntu7
ProcVersionSignature: Ubuntu 4.15.0-22.24-generic 4.15.17
Uname: Linux 4.15.0-22-generic x86_64
NonfreeKernelModules: wl nvidia_modeset nvidia
.proc.driver.nvidia.gpus.0000.0a.00.0: Error: [Errno 21] Is a directory: '/proc/driver/nvidia/gpus/0000:0a:00.0'
.proc.driver.nvidia.registry: Binary: ""
.proc.driver.nvidia.version:
 NVRM version: NVIDIA UNIX x86_64 Kernel Module 390.48 Thu Mar 22 00:42:57 PDT 2018
 GCC version: gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)
ApportVersion: 2.20.9-0ubuntu7
Architecture: amd64
BootLog: Error: [Errno 13] Permission denied: '/var/log/boot.log'
CompizPlugins: No value set for `/apps/compiz-1/general/screen0/options/active_plugins'
CompositorRunning: None
CurrentDesktop: ubuntu:GNOME
Date: Sun May 27 11:43:16 2018
DistUpgraded: 2018-05-21 10:52:35,978 DEBUG Running PostInstallScript: './xorg_fix_proprietary.py'
DistroCodename: bionic
DistroVariant: ubuntu
DkmsStatus:
 bcmwl, 6.30.223.271+bdcom, 4.15.0-20-generic, x86_64: installed
 bcmwl, 6.30.223.271+bdcom, 4.15.0-22-generic, x86_64: installed
 nvidia, 390.48, 4.15.0-20-generic, x86_64: installed
 nvidia, 390.48, 4.15.0-22-generic, x86_64: installed
EcryptfsInUse: Yes
ExtraDebuggingInterest: Yes
GraphicsCard:
 Intel Corporation HD Graphics 5500 [8086:1616] (rev 09) (prog-if 00 [VGA controller])
   Subsystem: Hewlett-Packard Company HD Graphics 5500 [103c:229d]
   Subsystem: Hewlett-Packard Company GM108M [GeForce 840M] [103c:229d]
InstallationDate: Installed on 2017-11-02 (205 days ago)
InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64 (20171018)
MachineType: Hewlett-Packard HP ENVY 17 Notebook PC
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-22-generic root=UUID=2c76a9d7-c435-4600-abc9-2ecd10fc3001 ro quiet splash vt.handoff=1SourcePackage: xorg
UpgradeStatus: Upgraded to bionic on 2018-05-21 (6 days ago)
dmi.bios.date: 11/28/2014
dmi.bios.vendor: Insyde
dmi.bios.version: F.33
dmi.board.asset.tag: Type2 - Board Asset Tag
dmi.board.name: 229D
dmi.board.vendor: Hewlett-Packard
dmi.board.version: 78.15
dmi.chassis.type: 10
dmi.chassis.vendor: Hewlett-Packard
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnInsyde:bvrF.33:bd11/28/2014:svnHewlett-Packard:pnHPENVY17NotebookPC:pvr0973120000405F10000620180:rvnHewlett-Packard:rn229D:rvr78.15:cvnHewlett-Packard:ct10:cvrChassisVersion:
dmi.product.family: 103C_5335KV G=N L=CON B=HP S=ENV X=Null
dmi.product.name: HP ENVY 17 Notebook PC
dmi.product.version: 0973120000405F10000620180
dmi.sys.vendor: Hewlett-Packard
version.compiz: compiz N/A
version.libdrm2: libdrm2 2.4.91-2
version.libgl1-mesa-dri: libgl1-mesa-dri 18.0.0~rc5-1ubuntu1
version.libgl1-mesa-glx: libgl1-mesa-glx 18.0.0~rc5-1ubuntu1
version.nvidia-graphics-drivers: nvidia-graphics-drivers-* N/A
version.xserver-xorg-core: xserver-xorg-core 2:1.19.6-1ubuntu4
version.xserver-xorg-input-evdev: xserver-xorg-input-evdev N/A
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:18.0.1-1
version.xserver-xorg-video-intel: xserver-xorg-video-intel 2:2.99.917+git20171229-1
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:1.0.15-2

Related branches

Revision history for this message
Yoshee (yoshee) wrote :
Timo Aaltonen (tjaalton)
affects: xorg (Ubuntu) → nvidia-graphics-drivers-390 (Ubuntu)
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in nvidia-graphics-drivers-390 (Ubuntu):
status: New → Confirmed
Changed in ubuntu-release-upgrader (Ubuntu):
status: New → Confirmed
Revision history for this message
James Gross (xeboc) wrote :

https://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-release-upgrader/trunk/view/head:/DistUpgrade/xorg_fix_proprietary.py#L112

apt-file search nvidia_drv.so
nvidia-340: /usr/lib/nvidia-340/xorg/nvidia_drv.so
xserver-xorg-video-nvidia-390: /usr/lib/x86_64-linux-gnu/nvidia/xorg/nvidia_drv.so
xserver-xorg-video-nvidia-legacy-340xx: /usr/lib/nvidia/legacy-340xx/nvidia_drv.so
xserver-xorg-video-nvidia-legacy-390xx: /usr/lib/nvidia/legacy-390xx/nvidia_drv.so

The file the release upgrader is looking for doesn't exist in the current nvidia packages or exists in another location.

Revision history for this message
Brian Murray (brian-murray) wrote :

Could you provide your /var/log/dist-upgrade/main.log file from the upgrade to Ubuntu 18.04? Thanks in advance.

Revision history for this message
Yoshee (yoshee) wrote :

Hi Brian, I attach the log file.

Revision history for this message
Brian Murray (brian-murray) wrote :

Looking at the log file we can see the following:

2018-05-21 09:55:56,514 DEBUG nvidiaUpdate()
2018-05-21 09:55:58,109 INFO no old nvidia driver installed, installing no new

Then looking at DistUpgradeCache from ubuntu-release-upgrader there is this:

 540 # get new detection module and use the modalises files
 541 # from within the release-upgrader
 542 nv = NvidiaDetection(obsolete="./ubuntu-drivers-obsolete.pkgs")
 543 #nv = NvidiaDetection()
 544 # check if a binary driver is installed now
 545 for oldDriver in nv.oldPackages:
 546 if oldDriver in self and self[oldDriver].is_installed:
 547 self.mark_remove(oldDriver, "old nvidia driver")
 548 break
 549 else:
 550 logging.info("no old nvidia driver installed, installing no new")
 551 return False

Unfortunately, the dist-upgrader tarball doesn't actually have "ubuntu-drivers-obsolete.pkgs" in it so nv.oldPackages is always empty and no old nvidia driver will be detected.

Revision history for this message
Brian Murray (brian-murray) wrote :

And it looks like this is something I removed last year, here's the changelog entry.

  * build-tarball.sh: Stop updating ubuntu-drivers-obsolete.pkgs since it
    isn't needed any more see Launchpad bug 1682291 for details.

Revision history for this message
Brian Murray (brian-murray) wrote :

Hey, thanks for helping out with the log file. Could you also add '/var/log/dist-upgrade/xorg_fixup.log' to your bug report? Thanks again!

tags: added: id-5c12a0777dd3062c173d4d7e
Revision history for this message
Yoshee (yoshee) wrote :
Revision history for this message
Brian Murray (brian-murray) wrote :

The xorg apport package hook checks /proc/modules for the presence of nvidia which seems better than looking multiple places for an nvidia .so file, so let's convert xorg_fix_proprietary.py to check for that.

Changed in ubuntu-release-upgrader (Ubuntu):
status: Confirmed → In Progress
importance: Undecided → High
assignee: nobody → Brian Murray (brian-murray)
Changed in nvidia-graphics-drivers-390 (Ubuntu):
status: Confirmed → Won't Fix
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:19.04.8

---------------
ubuntu-release-upgrader (1:19.04.8) disco; urgency=medium

  * debian/source_ubuntu-release-upgrader.py: Include xorg_fixup.log if it
    exists.
  * DistUpgrade/xorg_fix_proprietary.py: modify how the system is checked to
    see if nvidia is being used, drop fglrx check since it has been
    deprecated. (LP: #1773637)

 -- Brian Murray <email address hidden> Mon, 17 Dec 2018 14:04:03 -0800

Changed in ubuntu-release-upgrader (Ubuntu):
status: In Progress → Fix Released
Changed in nvidia-graphics-drivers-390 (Ubuntu Cosmic):
status: New → Won't Fix
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Yoshee, or anyone else affected,

Accepted ubuntu-release-upgrader into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:18.10.11.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-release-upgrader (Ubuntu Cosmic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-cosmic
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote :

I tested this upgrade process and it worked for me, setting to verification-done.

bdmurray@clean-bionic-amd64:/tmp/ubuntu-release-upgrader-0vo0ih0i$ grep "release-upgrader version" /var/log/dist-upgrade/main.log
2019-01-09 15:50:20,150 INFO release-upgrader version '18.10.11.3' started
bdmurray@clean-bionic-amd64:/tmp/ubuntu-release-upgrader-0vo0ih0i$ cat /etc/X11/xorg.conf
Section "Device"
    Identifier "nvidia"
EndSection
bdmurray@clean-bionic-amd64:/tmp/ubuntu-release-upgrader-0vo0ih0i$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.10

tags: added: verification-done verification-done-cosmic
removed: verification-needed verification-needed-cosmic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:18.10.11.3

---------------
ubuntu-release-upgrader (1:18.10.11.3) cosmic; urgency=medium

  * DistUpgrade/DistUpgradeCache.py: When calculating free space needed for
    mount points don't use a negative number as the buffer. (LP: #1786484)
  * fix more typos re RELEASE_UPRADER_ALLOW_THIRD_PARTY.
  * DistUpgrade/DistUpgradeController.py: When rewriting sources.list entries
    check to see if the source provides packages for the release to which the
    upgrade is occurring. If it doesn't the entry is disabled thereby
    improving upgrades with PPAs. (LP: #1807043)
  * do-release-upgrade: add a parameter to allow third party mirrors and
    repositories, additionally pass along the environmental variable
    RELEASE_UPGRADER_ALLOW_THIRD_PARTY via pkexec and sudo. (LP: #1807032)
  * DistUpgrade/DistUpgradeCache.py: improve messaging regarding failures to
    calculate the upgrade and suggest using ppa-purge if PPAs are an issue.
  * DistUpgrade/DistUpgradeCache.py: in the event there is a failure to
    calculate the upgrade provide information about the log files in
    /var/log/dist-upgrade. (LP: #1771387)
  * debian/source_ubuntu-release-upgrader.py: Include xorg_fixup.log if it
    exists.
  * DistUpgrade/xorg_fix_proprietary.py: modify how the system is checked to
    see if nvidia is being used, drop fglrx check since it has been
    deprecated. (LP: #1773637)

 -- Brian Murray <email address hidden> Thu, 20 Dec 2018 12:31:37 -0800

Changed in ubuntu-release-upgrader (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for ubuntu-release-upgrader has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Yoshee, or anyone else affected,

Accepted ubuntu-release-upgrader into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:18.04.30 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-release-upgrader (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-bionic
removed: verification-done
Revision history for this message
Brian Murray (brian-murray) wrote :

I tested this using the ubuntu-release-upgrader from -proposed when upgrading from Bionic to Cosmic. The nvidia "Driver" line was left in my /etc/X11/xorg.conf file after the upgrade completed.

bdmurray@clean-bionic-amd64:/tmp/ubuntu-release-upgrader-1aog5myp$ cat /etc/X11/xorg.conf
Section "Device"
            Identifier "nvidia"
            Driver "nvidia"
EndSection
bdmurray@clean-bionic-amd64:/tmp/ubuntu-release-upgrader-1aog5myp$ apt-cache policy ubuntu-release-upgrader-core | head -n3
ubuntu-release-upgrader-core:
  Installed: 1:18.10.11.3
  Candidate: 1:18.10.11.3
bdmurray@clean-bionic-amd64:/tmp/ubuntu-release-upgrader-1aog5myp$ grep "18.10.11.3" /var/log/dist-upgrade/main.log
2019-01-23 07:29:06,129 INFO release-upgrader version '18.10.11.3' started

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

The real test should be upgrading from Xenial to Bionic and I did that and the nvidia Driver section was still enabled.

bdmurray@clean-xenial-amd64:/tmp/ubuntu-release-upgrader-n06oth47$ cat /etc/X11/xorg.conf
Section "Device"
  Driver "nvidia"
EndSection
bdmurray@clean-xenial-amd64:/tmp/ubuntu-release-upgrader-n06oth47$ grep "release-upgrader" /var/log/dist-upgrade/main.log
2019-01-23 08:13:17,917 INFO release-upgrader version '18.04.29' started

The version is incorrectly listed in main.log as DistUpgradeVersion.py somehow wasn't updated during the build process.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-release-upgrader - 1:18.04.30

---------------
ubuntu-release-upgrader (1:18.04.30) bionic; urgency=medium

  * data/release-upgrades: Clarify documentation regarding the behavior for
    different Prompt settings. (LP: #1795024)
  * do-release-upgrade: Utilize information regarding what Prompt is set to so
    that a more informative error message can be displayed.
    (LP: #1798618, LP: #1795024)
  * DistUpgrade/DistUpgradeCache.py: When calculating free space needed for
    mount points don't use a negative number as the buffer. (LP: #1786484)
  * fix more typos re RELEASE_UPRADER_ALLOW_THIRD_PARTY
  * DistUpgrade/DistUpgradeController.py: When rewriting sources.list entries
    check to see if the source provides packages for the release to which the
    upgrade is occurring. If it doesn't the entry is disabled thereby
    improving upgrades with PPAs. (LP: #1807043)
  * do-release-upgrade: add a parameter to allow third party mirrors and
    repositories, additionally pass along the environmental variable
    RELEASE_UPGRADER_ALLOW_THIRD_PARTY via pkexec and sudo. (LP: #1807032)
  * DistUpgrade/DistUpgradeCache.py: improve messaging regarding failures to
    calculate the upgrade and suggest using ppa-purge if PPAs are an issue.
  * DistUpgrade/DistUpgradeCache.py: in the event there is a failure to
    calculate the upgrade provide information about the log files in
    /var/log/dist-upgrade. (LP: #1771387)
  * debian/source_ubuntu-release-upgrader.py: Include xorg_fixup.log if it
    exists.
  * DistUpgrade/xorg_fix_proprietary.py: modify how the system is checked to
    see if nvidia is being used, drop fglrx check since it has been
    deprecated. (LP: #1773637)

 -- Brian Murray <email address hidden> Wed, 16 Jan 2019 15:53:58 -0800

Changed in ubuntu-release-upgrader (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.