libguestfs cannot start the qemu process on s390x

Bug #1766534 reported by bugproxy
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Unassigned
libguestfs (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Won't Fix
Undecided
Unassigned
Artful
Won't Fix
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * the way libguestfs launches libvirt is not working on s390x

 * backport fixes to work on that platform as well

 * The TL;DR of the changes is to add arch specific options to the calls being made

[Test Case]

 * Note: IBM will assist in verifying those and already has on the PPA
 * To run yourself from the original description you can use libguestfs-test-tool:
  1. apt install libguestfs-tools python-libguestfs linux-image-generic qemu
  2. sudo libguestfs-test-tool
  This will by default break on e.g.:
    qemu-system-s390x: -device virtio-rng-pci,rng=rng0: MSI-X support is mandatory in the S390 architecture
    libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
  3. (upgrade from ppa or proposed depending on what you test)
  4. sudo libguestfs-test-tool
    [...]
    ===== TEST FINISHED OK =====

[Regression Potential]

 * This modifies how arguments are created, but it does so only for s390x so the regression should be limited to that architecture.
   The issue would be that it doesn't start properly, but since that is the current state there is no real regression - it would just replacve one way of failing for another - and since the tests by IBM on the PPA were good we are convinced that even that regression-risk would not trigger.

 * The FTBFS is a build time only change and should not trigger regressions later on.

[Other Info]

 * Along the fix requested this fixes an FTBFS in Bionic due to the move to a new default JDK version.

---

The libguestfs versions < 1.37.22 don't work on s390x, as they don't
contain the patches which include the s390x specifics to instruct qemu-kvm
correctly. Ubuntu 16.04 contains an older version than that, which makes
libguestfs not usable on s390x.

Steps to reproduce
==================
A chronological list of steps which will bring off the issue I noticed:

    mz@s390xhost$ docker run -it --privileged --rm ubuntu:xenial bash
    root@5dde0aef5d2e:/#
    root@5dde0aef5d2e:/#
    root@5dde0aef5d2e:/# apt update
    root@5dde0aef5d2e:/# DEBIAN_FRONTEND=noninteractive apt install -y \
    libguestfs-tools \
    python-libguestfs \
    linux-image-generic \
    qemu
    root@5dde0aef5d2e:/#
    root@5dde0aef5d2e:/#
    root@5dde0aef5d2e:/# export LIBGUESTFS_DEBUG=1
    root@5dde0aef5d2e:/# export LIBGUESTFS_TRACE=1
    root@5dde0aef5d2e:/# libguestfs-test-tool

Expected result
===============
The libguestfs-test-tool passes and I can use it on s390x.

To check if qemu on s390x works independently from libguestfs, I use this
command:

    root@5dde0aef5d2e:/# qemu-system-s390x \
    -enable-kvm \
    -nographic \
    -kernel /boot/vmlinuz-4.4.0-119-generic \
    -initrd /boot/initrd.img-4.4.0-119-generic \
    -m 1G \
    -M s390-ccw-virtio

    [...] # boot

    BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
    Enter 'help' for a list of built-in commands.

    (initramfs)

Actual result
=============

libguestfs instructs qemu to use pci devices and serial consoles
which doesn't work on s390x, e.g.:

    root@5dde0aef5d2e:/# libguestfs-test-tool
         ************************************************************
         * IMPORTANT NOTICE
         *
         * When reporting bugs, include the COMPLETE, UNEDITED
         * output below in your bug report.
         *
         ************************************************************
    libguestfs: trace: set_verbose true
    libguestfs: trace: set_verbose = 0
    libguestfs: trace: set_verbose true
    libguestfs: trace: set_verbose = 0
    LIBGUESTFS_DEBUG=1
    LIBGUESTFS_TRACE=1
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    SELinux: sh: 1: getenforce: not found
    libguestfs: trace: add_drive_scratch 104857600
    libguestfs: trace: get_tmpdir
    libguestfs: trace: get_tmpdir = "/tmp"
    libguestfs: trace: disk_create "/tmp/libguestfs82Gj1d/scratch.1" "raw" 104857600
    libguestfs: trace: disk_create = 0
    libguestfs: trace: add_drive "/tmp/libguestfs82Gj1d/scratch.1" "format:raw" "cachemode:unsafe"
    libguestfs: trace: add_drive = 0
    libguestfs: trace: add_drive_scratch = 0
    libguestfs: trace: get_append
    libguestfs: trace: get_append = "NULL"
    guestfs_get_append: (null)
    libguestfs: trace: get_autosync
    libguestfs: trace: get_autosync = 1
    guestfs_get_autosync: 1
    libguestfs: trace: get_backend
    libguestfs: trace: get_backend = "direct"
    guestfs_get_backend: direct
    libguestfs: trace: get_backend_settings
    libguestfs: trace: get_backend_settings = []
    guestfs_get_backend_settings: []
    libguestfs: trace: get_cachedir
    libguestfs: trace: get_cachedir = "/var/tmp"
    guestfs_get_cachedir: /var/tmp
    libguestfs: trace: get_direct
    libguestfs: trace: get_direct = 0
    guestfs_get_direct: 0
    libguestfs: trace: get_hv
    libguestfs: trace: get_hv = "/usr/bin/qemu-system-s390x"
    guestfs_get_hv: /usr/bin/qemu-system-s390x
    libguestfs: trace: get_memsize
    libguestfs: trace: get_memsize = 500
    guestfs_get_memsize: 500
    libguestfs: trace: get_network
    libguestfs: trace: get_network = 0
    guestfs_get_network: 0
    libguestfs: trace: get_path
    libguestfs: trace: get_path = "/usr/lib/s390x-linux-gnu/guestfs"
    guestfs_get_path: /usr/lib/s390x-linux-gnu/guestfs
    libguestfs: trace: get_pgroup
    libguestfs: trace: get_pgroup = 0
    guestfs_get_pgroup: 0
    libguestfs: trace: get_program
    libguestfs: trace: get_program = "libguestfs-test-tool"
    guestfs_get_program: libguestfs-test-tool
    libguestfs: trace: get_recovery_proc
    libguestfs: trace: get_recovery_proc = 1
    guestfs_get_recovery_proc: 1
    libguestfs: trace: get_selinux
    libguestfs: trace: get_selinux = 0
    guestfs_get_selinux: 0
    libguestfs: trace: get_smp
    libguestfs: trace: get_smp = 1
    guestfs_get_smp: 1
    libguestfs: trace: get_tmpdir
    libguestfs: trace: get_tmpdir = "/tmp"
    guestfs_get_tmpdir: /tmp
    libguestfs: trace: get_trace
    libguestfs: trace: get_trace = 1
    guestfs_get_trace: 1
    libguestfs: trace: get_verbose
    libguestfs: trace: get_verbose = 1
    guestfs_get_verbose: 1
    host_cpu: s390x
    Launching appliance, timeout set to 600 seconds.
    libguestfs: trace: launch
    libguestfs: trace: version
    libguestfs: trace: version = <struct guestfs_version *>
    libguestfs: trace: get_backend
    libguestfs: trace: get_backend = "direct"
    libguestfs: launch: program=libguestfs-test-tool
    libguestfs: launch: version=1.32.2
    libguestfs: launch: backend registered: unix
    libguestfs: launch: backend registered: uml
    libguestfs: launch: backend registered: libvirt
    libguestfs: launch: backend registered: direct
    libguestfs: launch: backend=direct
    libguestfs: launch: tmpdir=/tmp/libguestfs82Gj1d
    libguestfs: launch: umask=0022
    libguestfs: launch: euid=0
    libguestfs: trace: get_backend_setting "force_tcg"
    libguestfs: trace: get_backend_setting = NULL (error)
    libguestfs: trace: get_cachedir
    libguestfs: trace: get_cachedir = "/var/tmp"
    libguestfs: [00000ms] begin building supermin appliance
    libguestfs: [00000ms] run supermin
    libguestfs: command: run: /usr/bin/supermin
    libguestfs: command: run: \ --build
    libguestfs: command: run: \ --verbose
    libguestfs: command: run: \ --if-newer
    libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
    libguestfs: command: run: \ --copy-kernel
    libguestfs: command: run: \ -f ext2
    libguestfs: command: run: \ --host-cpu s390x
    libguestfs: command: run: \ /usr/lib/s390x-linux-gnu/guestfs/supermin.d
    libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
    supermin: version: 5.1.14
    supermin: package handler: debian/dpkg
    supermin: acquiring lock on /var/tmp/.guestfs-0/lock
    supermin: build: /usr/lib/s390x-linux-gnu/guestfs/supermin.d
    supermin: reading the supermin appliance
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/excludefiles type uncompressed excludefiles
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/hostfiles type uncompressed hostfiles
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages type uncompressed packages
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages-hfsplus type uncompressed packages
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages-reiserfs type uncompressed packages
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/packages-xfs type uncompressed packages
    supermin: build: visiting /usr/lib/s390x-linux-gnu/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
    supermin: mapping package names to installed packages
    supermin: resolving full list of package dependencies
    supermin: build: 194 packages, including dependencies
    supermin: build: 6751 files
    supermin: build: 3694 files, after matching excludefiles
    supermin: build: 3696 files, after adding hostfiles
    supermin: build: 3696 files, after removing unreadable files
    supermin: build: 3700 files, after munging
    supermin: kernel: picked kernel vmlinuz-4.4.0-119-generic
    supermin: kernel: picked modules path /lib/modules/4.4.0-119-generic
    supermin: kernel: kernel_version 4.4.0-119-generic
    supermin: kernel: modules /lib/modules/4.4.0-119-generic
    supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.s5hkw6ja/root'
    supermin: ext2: populating from base image
    supermin: ext2: copying files from host filesystem
    supermin: ext2: copying kernel modules
    supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.s5hkw6ja/initrd'
    supermin: ext2: wrote 9 modules to minimal initrd
    supermin: renaming /var/tmp/.guestfs-0/appliance.d.s5hkw6ja to /var/tmp/.guestfs-0/appliance.d
    libguestfs: [01090ms] finished building supermin appliance
    libguestfs: [01090ms] begin testing qemu features
    libguestfs: command: run: /usr/bin/qemu-system-s390x
    libguestfs: command: run: \ -display none
    libguestfs: command: run: \ -help
    libguestfs: command: run: /usr/bin/qemu-system-s390x
    libguestfs: command: run: \ -display none
    libguestfs: command: run: \ -version
    libguestfs: qemu version 2.5
    libguestfs: command: run: /usr/bin/qemu-system-s390x
    libguestfs: command: run: \ -display none
    libguestfs: command: run: \ -machine accel=kvm:tcg
    libguestfs: command: run: \ -device ?
    libguestfs: [01166ms] finished testing qemu features
    libguestfs: trace: get_backend_setting "gdb"
    libguestfs: trace: get_backend_setting = NULL (error)
    [01172ms] /usr/bin/qemu-system-s390x \
        -global virtio-blk-pci.scsi=off \
        -nodefconfig \
        -enable-fips \
        -nodefaults \
        -display none \
        -machine accel=kvm:tcg \
        -cpu host \
        -m 500 \
        -no-reboot \
        -rtc driftfix=slew \
        -global kvm-pit.lost_tick_policy=discard \
        -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
        -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
        -device virtio-scsi-pci,id=scsi \
        -drive file=/tmp/libguestfs82Gj1d/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
        -device scsi-hd,drive=hd0 \
        -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
        -device scsi-hd,drive=appliance \
        -device virtio-serial-pci \
        -serial stdio \
        -chardev socket,path=/tmp/libguestfs82Gj1d/guestfsd.sock,id=channel0 \
        -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
        -append 'panic=1 console=ttyS0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm'
    qemu-system-s390x: Warning: global kvm-pit.lost_tick_policy has invalid class name
    Alarm clock

Environment
===========

This affects ubuntu 16.04 (see below)

host
----

    $ lsb_release -a
    LSB Version: core-9.20160110ubuntu0.2-noarch:core-9.20160110ubuntu0.2-s390x:printing-9.20160110ubuntu0.2-noarch:printing-9.20160110ubuntu0.2-s390x:security-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-s390x
    Distributor ID: Ubuntu
    Description: Ubuntu 16.04.4 LTS
    Release: 16.04
    Codename: xenial

container
---------

    root@5dde0aef5d2e:/# lsb_release -a
    No LSB modules are available.
    Distributor ID: Ubuntu
    Description: Ubuntu 16.04.3 LTS
    Release: 16.04
    Codename: xenial

    root@5dde0aef5d2e:/# apt-cache policy libguestfs-tools
    libguestfs-tools:
      Installed: 1:1.32.2-4ubuntu2
      Candidate: 1:1.32.2-4ubuntu2
      Version table:
     *** 1:1.32.2-4ubuntu2 500
            500 http://ports.ubuntu.com/ubuntu-ports xenial/universe s390x Packages
            100 /var/lib/dpkg/status

Additional information
======================

Please be aware that there this is a known libguestfs upstream bug [1]
which is already fixed and backported by RedHat [2][3] and was the
trigger for a patch series on the libguestfs mailing list [4]. As an
example, the latest known needed patch is [5]. As a smoke test, in case
the libguestfs-test-tool result was a false positive, I used the python
binding code in Appendix A below, which is based on [6].

FWIW, I checked Ubuntu 18.04 (beta), and it doesn't work there too.
The packaged version of libguestfs is v1.36 in 18.04 bionic beaver,
which is too old (and/or doesn't contain the needed backports).

References
==========
[1] Bug 1376547 - qemu-system-s390x: -device isa-serial,chardev=charserial0,
    id=serial0: 'isa-serial' is not a valid device model name
    https://bugzilla.redhat.com/show_bug.cgi?id=1376547
[2] [Private] Bug 1479526 - [RFE] libguestfs for z Systems
    https://bugzilla.redhat.com/show_bug.cgi?id=1479526
[3] The RHEL7.5 branch of libguestfs with backports to v1.36
    https://github.com/libguestfs/libguestfs/tree/rhel-7.5
[4] The patch series of libguestfs on s390x:
    https://www.redhat.com/archives/libguestfs/2017-May/msg00066.html
[5] Fix of an x86 specific item to enable s390x in v1.37.22
    https://github.com/libguestfs/libguestfs/commit/5b60dd4eff02f48d344bcdad0d4bad4676ca9168
[6] https://rwmj.wordpress.com/2010/02/01/libguestfs-python-example/

Appendix A
==========

Smoke test, based on the example from the author of libguestfs [6]:

    root@26248e573a87:/# cat smoke.py
    #!/usr/bin/python

    # To see the Python docs, do: help (guestfs)
    # To see the general docs http://libguestfs.org/guestfs.3.html
    # For FAQ see http://libguestfs.org/FAQ.html
    import os
    import guestfs
    g = guestfs.GuestFS()
    f = open("/tmp/test.img", "w")
    f.truncate(500 * 1024 * 1024)
    f.close()
    g.add_drive("/tmp/test.img")
    g.launch()
    g.part_disk("/dev/sda", "mbr")
    g.mkfs("ext2", "/dev/sda1")
    g.mount("/dev/sda1", "/")
    f = open("/tmp/ks.cfg", "w")
    f.write("# this would be the kickstart file\n")
    f.write("#\n")
    f.write ("# end of file\n")
    f.close()
    g.upload("/tmp/ks.cfg", "/ks.cfg")
    g.sync()
    g.umount_all()

== Comment: #1 - Markus Zoeller <email address hidden> - 2018-04-20 06:51:10 ==
With the Debian Sid repository I could install libguestfs v1.38 in an
Ubuntu 18.04 (bionic beaver) container, all on s390x:

    [mz@s390xhost]$ docker run -it --privileged --rm s390x/ubuntu:18.04 bash
    $ apt update
    $ apt install -y vim
    $ apt-cache policy libguestfs-tools
    $ apt install -y software-properties-common \
    debian-archive-keyring \
    dirmngr
    $ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B48AD6246925553
    $ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 7638D0442B90D010
    $ add-apt-repository "deb http://ftp.de.debian.org/debian sid main"

    $ cat << EOF > /etc/apt/preferences.d/libguestfs.pref
    # Note: 2 blank lines are required between entries
    Package: *
    Pin: release a=bionic
    Pin-Priority: 500

    Package: *
    Pin: origin "ftp.de.debian.org"
    Pin-Priority: 300

    # Pattern includes 'python-guestfs' and 'libguestfs-tools' and similarily named dependencies
    Package: *guestfs*
    Pin: origin "ftp.de.debian.org"
    Pin-Priority: 700
    EOF
    $ cat /etc/apt/preferences.d/libguestfs.pref

    $ apt update
    $ apt-cache policy libguestfs-tools

    $ DEBIAN_FRONTEND=noninteractive apt install -y \
        libguestfs-tools \
        python-libguestfs \
        linux-image-generic \
        qemu-system \
        qemu-kvm

    $ apt-cache policy libguestfs-tools
    $ export LIBGUESTFS_DEBUG=1
    $ export LIBGUESTFS_TRACE=1
    $ libguestfs-test-tool
    [...]
    ===== TEST FINISHED OK =====

This verifies that libguestfs v1.38 has the needed code to work on s390x.

Related branches

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-166979 severity-high targetmilestone-inin1604
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Given the current Final Freeze for 18.04 this will (if done) have to be an SRU for 18.04 as well.
I just did the initial triage, but will leave it for the screener team to ensure it is as needed.

Changed in libguestfs (Ubuntu):
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
no longer affects: linux (Ubuntu)
Changed in ubuntu-z-systems:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
Frank Heimes (fheimes)
tags: added: universe
Changed in ubuntu-z-systems:
status: New → Triaged
importance: Undecided → High
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-05-22 04:05 EDT-------
@Canonical, any updates for this LP? Thx

Changed in ubuntu-z-systems:
assignee: Skipper Bug Screeners (skipper-screen-team) → Canonical Foundations Team (canonical-foundations)
tags: added: id-5b04b5dd6052dc18608ce46a
Revision history for this message
Frank Heimes (fheimes) wrote :

cosmic has 1.38 in proposed, should migrate into release pocket eventually.

Changed in libguestfs (Ubuntu):
status: New → Fix Committed
bugproxy (bugproxy)
tags: removed: bugnameltc-166979 id-5b04b5dd6052dc18608ce46a severity-high universe
tags: added: id-5b04b5dd6052dc18608ce46a
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

@cpaelzer
Once this lands in cosmic, are you going to work on SRUing this into stable series? Or is this actually out of scope?

Changed in ubuntu-z-systems:
assignee: Canonical Foundations Team (canonical-foundations) → nobody
Changed in libguestfs (Ubuntu Bionic):
assignee: Skipper Bug Screeners (skipper-screen-team) → nobody
Changed in libguestfs (Ubuntu):
assignee: Skipper Bug Screeners (skipper-screen-team) → nobody
assignee: nobody →  Christian Ehrhardt  (paelzer)
Changed in libguestfs (Ubuntu Xenial):
status: New → Incomplete
Changed in libguestfs (Ubuntu Artful):
status: New → Incomplete
Changed in libguestfs (Ubuntu Bionic):
status: New → Incomplete
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi xnox,
I had not planned/worked on SRU considerations on this, as usual it is a matter of applicability of the changes and the Cosmic portion actually migrating.
I think step one is to check what it waits for anyway.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

https://launchpad.net/ubuntu/+source/libguestfs/1:1.38.1-1ubuntu2 is in Cosmic.
So Cosmic is done.

I'll check if Bionic is trivial or not and then report back here, the older ones are out of scope IMHO unless IBM provides the backports needed.

Changed in libguestfs (Ubuntu):
status: Fix Committed → Fix Released
Changed in libguestfs (Ubuntu Xenial):
status: Incomplete → Won't Fix
Changed in libguestfs (Ubuntu Artful):
status: Incomplete → Won't Fix
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-07-17 05:18 EDT-------
Canonical, Bug occurred on Xenial. Therefore please provide an SRU to Xenial and Bionic

tags: added: bugnameltc-166979 severity-high
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Heinz o/,
the "bug occurred in Xenial" IMHO is that s390x support was not working at all back then, not a bug in the usual sense. I'd clearly call that a feature more than a bug.
I'll check doability back to 1.32, but I doubt it will be easy so many versions back. If you or someone else already have a set of patches for 1.32 that are known to be good let us know.
The RHEL 7.3 branch was 1.32 based but that has no support for it either.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Of the references:
- https://github.com/libguestfs/libguestfs/commit/5b60dd4eff02f48d344bcdad0d4bad4676ca9168
  Already is in 1.36.6
- https://www.redhat.com/archives/libguestfs/2017-May/msg00066.html
  Is also a part of the rh7.5 backports
- The RH7.5 backports are direct cherry picks

The list of picks from upstream needed should be
6328f332 applies as-is
ade2652b using rh7.5 BP f774a2ef
5fc76d6d applies as-is
fa6bc0fd not-needed (comment only)
ddde6f9e using rh7.5 BP 516f008f
93af83de applies as-is
a5bd493e using rh7.5 BP eddfb0a7, but we have not reverted 224de20 therefore slight noise using both patches as needed.

I currently iterate over building this in [1], but it has some issues for now (unrelated to the changes this has a FTBFS in Bionic atm).
The changes can be seen in the Merge Proposal [2].

I'll need to resolve the FTBFS first ...

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3323

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FTBFS is due to http://openjdk.java.net/jeps/313 which makes jdk >=10 unusable for the libguestfs code that is in Bionic.
Also see https://bugs.java.com/view_bug.do?bug_id=JDK-8182758
Fixed by https://github.com/libguestfs/libguestfs/commit/662dc5d0bf65e72dab11aa58d4bc373b5a3b7e75
Forked bug 1782138 for that.
With the above fixed I have a PPA ready.

Before going on with the SRU processing I'd ask IBM to verify from the PPA [1] if that would suffice as backport for Bionic - therefore keeping incomplete on the Bionic Task until confirmed.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3323

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I checked if the patches apply to 1.32 in Xenial, but it really is very different code.
If it is really important and one provides backports it can be considered.

But for the time being I still consider the support of this in Xenial a Feature more than a bug and the risk of breaking things with a hacky non-applying backport bigger than the gain.

For now let us please focus on Bionic and please verify that the PPA would work for you as expected.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

@hws

The package in question is in Universe, and not explicitly covered. Server team member is attempting to provide SRU(s) to enable this feature on a wishlist priority / good-will basis only. How far back it is reasonable and easy to backport this, is entirely the Canonical Server team call.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

@paelzer

Good catch on the java bug, especially since we will move to java 11.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

MP ack would be ready to go.
To make it clear that I wait on a pre-check before SRU processing here quoting my former comment:

Before going on with the SRU processing I'd ask IBM to verify from the PPA [1] if that would suffice as backport for Bionic - therefore keeping incomplete on the Bionic Task until confirmed.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3323

Revision history for this message
bugproxy (bugproxy) wrote :
Download full text (3.4 KiB)

------- Comment From <email address hidden> 2018-07-25 04:36 EDT-------
(In reply to comment #16)
> MP ack would be ready to go.
> To make it clear that I wait on a pre-check before SRU processing here
> quoting my former comment:
>
> Before going on with the SRU processing I'd ask IBM to verify from the PPA
> [1] if that would suffice as backport for Bionic - therefore keeping
> incomplete on the Bionic Task until confirmed.
>
> [1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3323

Summary
=======
The code in the PPA archive passes for Ubuntu 18.04 (see below).
The PPA archive does not seem to be appliable to Ubuntu 16.04 (see below).

Steps to reproduce (18.04)
==========================
mz@s390xhost$ docker run -it --privileged --rm ubuntu:18.04 bash
root@f1f7c829b38e:/#
root@f1f7c829b38e:/#
root@f1f7c829b38e:/# apt update
root@f1f7c829b38e:/# apt-get install -y software-properties-common
root@f1f7c829b38e:/# add-apt-repository -y ppa:ci-train-ppa-service/3323
root@f1f7c829b38e:/#
root@f1f7c829b38e:/# DEBIAN_FRONTEND=noninteractive apt install -y \
libguestfs-tools \
python-libguestfs \
linux-image-generic \
qemu
root@f1f7c829b38e:/#
root@f1f7c829b38e:/# apt-cache policy libguestfs-tools
libguestfs-tools:
Installed: 1:1.36.13-1ubuntu3.1~ppa3
Candidate: 1:1.36.13-1ubuntu3.1~ppa3
Version table:
*** 1:1.36.13-1ubuntu3.1~ppa3 500
500 http://ppa.launchpad.net/ci-train-ppa-service/3323/ubuntu bionic/main s390x Packages
100 /var/lib/dpkg/status
1:1.36.13-1ubuntu3 500
500 http://ports.ubuntu.com/ubuntu-ports bionic/universe s390x Packages
root@f1f7c829b38e:/#
root@f1f7c829b38e:/#
root@f1f7c829b38e:/#
root@f1f7c829b38e:/# export LIBGUESTFS_DEBUG=1
root@f1f7c829b38e:/# export LIBGUESTFS_TRACE=1
root@f1f7c829b38e:/#
root@f1f7c829b38e:/#
root@f1f7c829b38e:/#
root@f1f7c829b38e:/# libguestfs-test-tool
[...]
/usr/bin/qemu-system-s390x \
-global virtio-blk-ccw.scsi=off \
-enable-fips \
-nodefaults \
-display none \
-machine accel=kvm:tcg \
-cpu host \
-m 500 \
-no-reboot \
-rtc driftfix=slew \
-kernel /var/tmp/.guestfs-0/appliance.d/kernel \
-initrd /var/tmp/.guestfs-0/appliance.d/initrd \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-ccw,rng=rng0 \
-device virtio-scsi-ccw,id=scsi \
-drive file=/tmp/libguestfsC6T7tY/scratch1.img,cache=unsafe,format=raw,id=hd0,if=none \
-device scsi-hd,drive=hd0 \
-drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \
-device scsi-hd,drive=appliance \
-device virtio-serial-ccw \
[...]
===== TEST FINISHED OK =====

NOTE:
Please take note of the `ccw` devices here, instead of the `pci` devices
of the original bug description.

Steps to reproduce (16.04)
==========================
mz@s390xhost$ docker run -it --privileged --rm ubuntu:16.04 bash
root@c0cfb3274e1a:/#
root@c0cfb3274e1a:/#
root@c0cfb3274e1a:/# apt update
root@c0cfb3274e1a:/# apt-get install -y software-properties-common
root@c0cfb3274e1a:/# add-apt-repository -y ppa:ci-train-ppa-service/3323
root@c0cfb3274e1a:/#
root@c0cfb3274e1a:/# DEBIAN_FRONTEND=noninteractive apt install -y \
libguestfs-tools \
python-libguestfs \
linux-image-generic \
qemu
root@c0cfb3274e1a:/#
root@c0c...

Read more...

Frank Heimes (fheimes)
Changed in libguestfs (Ubuntu Bionic):
status: Incomplete → In Progress
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
description: updated
Changed in libguestfs (Ubuntu):
assignee:  Christian Ehrhardt  (paelzer) → nobody
Revision history for this message
Steve Langasek (vorlon) wrote :

> * TL;DR: install and use libguestfs-tools on s390x

No, please include the complete test case in the bug description.

Changed in libguestfs (Ubuntu Bionic):
status: In Progress → Incomplete
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Steve,
while IBM wanted to do the testing I can agree that this was too short in the Template for the SRU Team. The description wasn't far away thou, as I said in the template "IBM outlined ...", so this is from the comments that already were in the bug and description.

I made sure by testing against the PPA that this works as expected.

(one way to) reproduce:
1. apt install libguestfs-tools python-libguestfs linux-image-generic qemu
2. sudo libguestfs-test-tool
This will by default break on e.g.:
  qemu-system-s390x: -device virtio-rng-pci,rng=rng0: MSI-X support is mandatory in the S390 architecture
  libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
3. (upgrade from ppa or proposed depending on what you test)
4. sudo libguestfs-test-tool
[...]
===== TEST FINISHED OK =====

I hope this better description will suffice, otherwise let me know.
I'll add it to the SRU template and set back to in progress to wait for re-review.

Changed in libguestfs (Ubuntu Bionic):
status: Incomplete → In Progress
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Sorry, looking back I should have just said "see below" (as everything was there) instead of "run and use it" that would have been much better. Hope it is better now

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted libguestfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libguestfs/1:1.36.13-1ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in libguestfs (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Please make sure that any verification is performed on the packages in the main archive (bionic-proposed), not the PPA. Remember about adding package version information to the test results.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Download full text (35.4 KiB)

Upgrade without issue:

$ sudo apt install libguestfs-dev libguestfs-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libconfig-general-perl openvswitch-common
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libguestfs-hfsplus libguestfs-perl libguestfs-reiserfs libguestfs-xfs libguestfs0 python-guestfs
Suggested packages:
  libguestfs-gfs2 libguestfs-jfs libguestfs-nilfs libguestfs-rescue libguestfs-rsync libguestfs-zfs
The following packages will be upgraded:
  libguestfs-dev libguestfs-hfsplus libguestfs-perl libguestfs-reiserfs libguestfs-tools libguestfs-xfs libguestfs0
  python-guestfs
8 upgraded, 0 newly installed, 0 to remove and 32 not upgraded.
Need to get 8.437 kB of archives.
After this operation, 1.024 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x python-guestfs s390x 1:1.36.13-1ubuntu3.1 [158 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs-tools s390x 1:1.36.13-1ubuntu3.1 [4.916 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs-perl s390x 1:1.36.13-1ubuntu3.1 [292 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs-xfs s390x 1:1.36.13-1ubuntu3.1 [1.768 B]
Get:5 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs-reiserfs s390x 1:1.36.13-1ubuntu3.1 [1.780 B]
Get:6 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs-hfsplus s390x 1:1.36.13-1ubuntu3.1 [1.768 B]
Get:7 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs-dev s390x 1:1.36.13-1ubuntu3.1 [849 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports bionic-proposed/universe s390x libguestfs0 s390x 1:1.36.13-1ubuntu3.1 [2.216 kB]
Fetched 8.437 kB in 1s (9.945 kB/s)
(Reading database ... 165155 files and directories currently installed.)
Preparing to unpack .../0-python-guestfs_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking python-guestfs (1:1.36.13-1ubuntu3.1) over (1:1.36.13-1ubuntu3) ...
Preparing to unpack .../1-libguestfs-tools_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking libguestfs-tools (1:1.36.13-1ubuntu3.1) over (1:1.36.13-1ubuntu3) ...
Preparing to unpack .../2-libguestfs-perl_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking libguestfs-perl (1:1.36.13-1ubuntu3.1) over (1:1.36.13-1ubuntu3) ...
Preparing to unpack .../3-libguestfs-xfs_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking libguestfs-xfs:s390x (1:1.36.13-1ubuntu3.1) over (1:1.36.13-1ubuntu3) ...
Preparing to unpack .../4-libguestfs-reiserfs_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking libguestfs-reiserfs:s390x (1:1.36.13-1ubuntu3.1) over (1:1.36.13-1ubuntu3) ...
Preparing to unpack .../5-libguestfs-hfsplus_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking libguestfs-hfsplus:s390x (1:1.36.13-1ubuntu3.1) over (1:1.36.13-1ubuntu3) ...
Preparing to unpack .../6-libguestfs-dev_1%3a1.36.13-1ubuntu3.1_s390x.deb ...
Unpacking libguestfs-dev:s390x (...

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libguestfs - 1:1.36.13-1ubuntu3.1

---------------
libguestfs (1:1.36.13-1ubuntu3.1) bionic; urgency=medium

  * d/p/lp-1766534-s390x-*: backport s390x/CCW support from 1.37.22 to Bionic
    (LP: #1766534)
  * d/p/lp-1782138-java-support-OpenJDK-10.patch: Fix FTBFS due to newer
    jdk >=10 being in Bionic (LP: #1782138)

 -- Christian Ehrhardt <email address hidden> Tue, 17 Jul 2018 10:34:31 +0200

Changed in libguestfs (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for libguestfs has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-08-20 07:08 EDT-------
IBM Bugzilla status -> closed, Fix Released in Cosmis and Bionic

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.