Comment 11 for bug 1673350

Steve,

Can you please assign to Foundations (request from IBM is cyphermox).

Thanks.

                         Michael

On 03/16/2017 12:09 AM, Launchpad Bug Tracker wrote:
> bugproxy (bugproxy) has assigned this bug to you for Ubuntu:
>
> == Comment: #0 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:01:18 ==
> ---Problem Description---
> dm-queue-length module is not included in installer/initramfs
>
> On Ubuntu, multipath devices using the 'queue-length' path selector
> are non-functional on both the installer and initramfs environments;
> because the 'dm-queue-length' kernel module is not included in them.
>
> The multipath-modules.udeb (src:linux) does not include it in the installer,
> nor multipath-tools-boot (src:multipath-tools) installs it in the initramfs.
>
> One example is the EMC XtremIO storage, which has 'queue-length' defined as
> its path selector in the default multipath configuration, at least on 16.04.
>
> Other products may be affected if they are manually configured to use that
> path selector (e.g., via /etc/multipath.conf), and the mere switch of that
> might render the system _unbootable_ if booting from multipath, since the
> initramfs is affected.
>
> More recently this and another storage changed default path selectors out
> of 'queue-length', however, it's virtually possible for any storage system
> to be affected, with the described manual configuration change. So, this
> change is also desired on for the next stable release, 17.04, and later.
>
> Patches are provided for 16.04 and 17.04.
>
>
> Error logs:
> ----------
>
> This error log is seen in the installer, and multipath devices are not
> created for such storage system:
>
> ...
> Mar 14 06:41:54 kernel: [ 0.000000] Kernel command line: tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false --- quiet /install disk-detect/multipath/enable=true
> ...
> Mar 14 06:47:09 main-menu[3743]: INFO: Menu item 'disk-detect' selected
> ...
> Mar 14 06:47:14 anna-install: Installing multipath-udeb
> ...
> Mar 14 06:47:16 kernel: [ 324.968027] device-mapper: table: 252:0: multipath: unknown path selector type
> Mar 14 06:47:16 kernel: [ 324.968030] device-mapper: ioctl: error adding target to table
> ...
> Mar 14 06:47:17 kernel: [ 326.484059] device-mapper: table: 252:0: multipath: unknown path selector type
> Mar 14 06:47:17 kernel: [ 326.484062] device-mapper: ioctl: error adding target to table
> Mar 14 06:47:17 disk-detect: Mar 14 06:47:16 | mpatha: ignoring map
> ...
> Mar 14 06:47:17 disk-detect: Mar 14 06:47:17 | mpathg: ignoring map
> Mar 14 06:47:18 disk-detect: No multipath devices detected
>
>
> And after installing the system in an individual/single path device,
> and trying to switch to multipath via multipath-tools-boot, it also
> happens in the initramfs:
>
> Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
> Begin: Running /scripts/local-premount ... Begin: Waiting for udev to settle (multipath) ... done.
> [ 89.709480] device-mapper: table: 252:0: multipath: unknown path selector type
> [ 89.709537] device-mapper: ioctl: error adding target to table
> [ 89.756251] device-mapper: table: 252:0: multipath: unknown path selector type
> [ 89.837132] device-mapper: ioctl: error adding target to table
> [ 89.912346] device-mapper: table: 252:0: multipath: unknown path selector type
> [ 90.001719] device-mapper: ioctl: error adding target to table
> [ 90.300203] device-mapper: table: 252:0: multipath: unknown path selector type
> [ 90.303521] device-mapper: ioctl: error adding target to table
> Begin: Running /scripts/local-block ... done.
> Begin: Running /scripts/local-block ... done.
> Begin: Running /scripts/local-block ... done.
> ...
> Begin: Running /scripts/local-block ... done.
> done.
>
> Gave up waiting for root device. Common problems:
> - Boot args (cat /proc/cmdline)
> - Check rootdelay= (did the system wait long enough?)
> - Check root= (did the system wait for the right device?)
> - Missing modules (cat /proc/modules; ls /dev)
> ALERT! UUID=0568fc48-2921-4ba2-b1b5-127c1d5a310e does not exist. Dropping to a shell!
>
>
> BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
> Enter 'help' for a list of built-in commands.
>
> (initramfs)
>
> Contact Information = <email address hidden>
>
> ---uname output---
> 4.4.0-31, 4.10.0-11
>
> Machine Type = N/A
>
> ---boot type---
> CDROM / ISO image
>
> ---Install repository type---
> CDROM
>
> ---Point of failure---
> Other failure during installation (stage 1)
>
> == Comment: #1 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:12:30 ==
> Patch for 16.04 multipath-tools
>
> == Comment: #2 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:12:56 ==
> Patch for 16.04 linux
>
> == Comment: #3 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:13:18 ==
> Patch for 17.04 multipath-tools
>
> == Comment: #4 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:16:27 ==
> Patch verification on 16.04:
> ---------------------------
>
> multipath-tools source package:
>
> # sed -n '/vendor.*XtremIO/,/selector/ p' libmultipath/hwtable.c
> .vendor = "XtremIO",
> .product = "XtremApp",
> .features = DEFAULT_FEATURES,
> .hwhandler = DEFAULT_HWHANDLER,
> .selector = "queue-length 0",
>
> - initramfs:
> ---------
>
> Before (module not included in initramfs):
>
> # dpkg -s multipath-tools-boot | grep ^Version:
> Version: 0.5.0+git1.656f8865-5ubuntu2.4
>
> # lsinitramfs /boot/initrd.img | grep dm-queue-length
> #
>
>
> After (module is included in initramfs):
>
> # dpkg -i multipath-tools*.deb kpartx*.deb
>
> # dpkg -s multipath-tools-boot | grep ^Version:
> Version: 0.5.0+git1.656f8865-5ubuntu2.4dmqueuelength1
>
> # lsinitramfs /boot/initrd.img | grep dm-queue-length
> lib/modules/4.4.0-66-generic/kernel/drivers/md/dm-queue-length.ko
>
>
> - kernel udeb:
> -----------
>
> # dpkg-deb -c multipath-modules-4.4.0-66-generic-di_4.4.0-66.87dmqueuelength1_ppc64el.udeb | grep dm-queue-length
> -rw-r--r-- root/root 12174 2017-03-14 17:17 ./lib/modules/4.4.0-66-generic/kernel/drivers/md/dm-queue-length.ko
>
>
> Patch verification on 17.04:
> ---------------------------
>
> Before:
>
> # dpkg -s multipath-tools-boot | grep ^Version:
> Version: 0.6.4-3ubuntu1
>
> # lsinitramfs /boot/initrd.img | grep dm-queue-length
> #
>
> After:
>
> # dpkg -i multipath-tools*.deb kpartx*.deb
>
> # dpkg -s multipath-tools-boot | grep ^Version:
> Version: 0.6.4-3ubuntu1dmqueuelength1
>
> # lsinitramfs /boot/initrd.img | grep dm-queue-length
> lib/modules/4.10.0-11-generic/kernel/drivers/md/dm-queue-length.ko
>
>
> - kernel udeb:
> -----------
>
> # dpkg-deb -c multipath-modules-4.10.0-11-generic-di_4.10.0-11.13dmqueuelength1_ppc64el.udeb | grep dm-queue-length
> -rw-r--r-- root/root 13128 2017-03-15 11:14 ./lib/modules/4.10.0-11-generic/kernel/drivers/md/dm-queue-length.ko
>
> == Comment: #5 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:20:14 ==
> Patch for 17.04 linux
>
> == Comment: #9 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:24:40 ==
> @taco-screen-team
>
> Please assign this bug to @cyphermox for the multipath-tools components, at least, as a suggestion.
> He's handled most of the multipath-tools related bugs/patches we've been contributing with.
>
> Thank you.
>
> == Comment: #10 - Mauricio Faria De Oliveira <email address hidden> - 2017-03-15 09:40:17 ==
> It's possible to reproduce this problem in a qemu-kvm guest,
> with an emulated disk: force the queue-length path selector.
>
> # name=mfo-1704
> # disk=/var/lib/libvirt/images/$name.qcow2
> # iso=/var/lib/libvirt/images/zesty-server-ppc64el.iso.2017-03-14
>
> # qemu-img create -f qcow2 $disk 128g
>
> # virt-install \
> --name $name \
> --cdrom $iso \
> --vcpus 8,sockets=1,cores=1,threads=8 \
> --memory 8192 \
> --controller type=scsi,model=virtio-scsi \
> --disk format=qcow2,path=$disk \
> --disk device=cdrom,readonly=true,path=$iso \
> --network bridge=virbr0,model=virtio
>
>
> Before the disk-detection stage (e.g., set-up users and passwords),
> Go Back, Execute a shell, and set up multipath.conf, and resume install:
>
> ~ # cat <<EOF >/etc/multipath.conf
> defaults {
> path_selector "queue-length 0"
> user_friendly_names yes
> find_multipaths no
> }
> EOF
>
> ~ # exit
>
> ** Affects: ubuntu
> Importance: Undecided
> Assignee: Taco Screen team (taco-screen-team)
> Status: New
>
>
> ** Tags: architecture-ppc64le bugnameltc-152598 severity-critical targetmilestone-inin16043

--
Michael Hohnbaum
OIL Program Manager
Power (ppc64el) Development Project Manager
Canonical, Ltd.