fail to activate fcp device for installation

Bug #1596968 reported by bugproxy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
Medium
Dimitri John Ledkov
s390-zfcp (Ubuntu)
Fix Released
Wishlist
Skipper Bug Screeners
Xenial
Fix Released
Wishlist
Dimitri John Ledkov

Bug Description

[Test case]
* Start installation with many zfcp devices >>100
* It should be able to complete zfcp device activation

== Comment: #0 - XIANG ZHANG - 2016-06-20 09:28:02 ==
---Problem Description---
Can not activate FCP devices for installation

---uname output---
Linux ilablnx3 4.4.0-24-generic #43-Ubuntu SMP Wed Jun 8 19:26:02 UTC 2016 s390x s390x s390x GNU/Linux

Machine Type = z13 lpar

---boot type---
Network boot

---Install repository type---
Local repository

---Install repository Location---
9.11.56.168:/OS_IMAGE/ISO/ubuntu/s390

---Point of failure---
Other failure during installation (stage 1)

I am installing a sanboot s390 ubuntu server via ftp server on a native lpar. But I am not able to activate fcp device for installation. However I am able active the fcp device and attach san boot lun via shell.

echo 1 > /sys/bus/ccw/driver/zfcp/0.0.1800/online
echo 0x0000000000000000 > /sys/bus/ccw/driver/zfcp/0.0.1800/0x5005076802401ee8/unit_add

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-142843 severity-medium targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → debian-installer (Ubuntu)
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Please provide installer logs, and the installer version that was in use. E.g. /var/log/syslog from the d-i environment, or /var/log/installer directory from the installed system.

Changed in debian-installer (Ubuntu):
status: New → Incomplete
importance: Undecided → Wishlist
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in debian-installer (Ubuntu):
status: Incomplete → New
Revision history for this message
Kevin W. Rudd (kevinr) wrote :
Revision history for this message
Kevin W. Rudd (kevinr) wrote :
Revision history for this message
Kevin W. Rudd (kevinr) wrote :
Revision history for this message
Kevin W. Rudd (kevinr) wrote :

Sorry about the attachment delay. We are experiencing bridge issues, so the mirror is currently broken.

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-06-28 12:31 EDT-------
(In reply to comment #21)
> Please provide installer logs, and the installer version that was in use.
> E.g. /var/log/syslog from the d-i environment, or /var/log/installer
> directory from the installed system.

I would love to, but we seem to have experienced some bridge hiccups today, so the attachments were not passed over, and I have no idea what LP bug number was assigned to the mirror. Please document the LP number, and I'll manually attach the data.

Thanks.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Hello,

After filing a bug report you can search for it on launchpad by arch tag:
E.g. https://bugs.launchpad.net/ubuntu/+bugs?field.tag=architecture-s39064

There is always internal bug number as tag, e.g. to find this bug using LTC-142843:
https://bugs.launchpad.net/ubuntu/+bugs?field.tag=bugnameltc-142843

The attachments seemed to have come through now.

Regards,

Dimitri.

no longer affects: debian-installer
affects: debian-installer (Ubuntu) → s390-zfcp (Ubuntu)
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

From the logs:

After all the devices are added, and there is no pressed, and 0.0.19fd device is selected the s390-zfcp component bombs out:

Jun 20 08:39:17 s390-zfcp[5708]: DEBUG: DETECT: Added FCP device: 0.0.19fd: online=0 npiv=0
Jun 20 08:39:17 s390-zfcp[5708]: DEBUG: DETECT: Automatic LUN scanning is enabled
Jun 20 08:39:17 s390-zfcp[5708]: DEBUG: PRESEED: No preseed data available
Jun 20 08:39:18 s390-zfcp[5708]: DEBUG: SELECT: Using FCP device 0.0.1800
Jun 20 08:39:18 main-menu[346]: WARNING **: Configuring 's390-zfcp' failed with error code 139
Jun 20 08:39:18 main-menu[346]: WARNING **: Menu item 's390-zfcp' failed.
Jun 20 08:39:18 kernel: [ 220.868144] User process fault: interruption code 0038 ilc:3 in libc.so.6[3ffa2200000+185000]
Jun 20 08:39:18 kernel: [ 220.868153] failing address: 2c302e302e313000 TEID: 2c302e302e313800
Jun 20 08:39:18 kernel: [ 220.868155] Fault in primary space mode while using user ASCE.
Jun 20 08:39:18 kernel: [ 220.868157] AS:00000001eeebc1c7 R3:0000000000000024
Jun 20 08:39:18 kernel: [ 220.868161] CPU: 1 PID: 5708 Comm: zfcp-config Tainted: G E 4.4.0-21-generic #37-Ubuntu
Jun 20 08:39:18 kernel: [ 220.868162] task: 00000001f197d780 ti: 00000001f0d58000 task.ti: 00000001f0d58000
Jun 20 08:39:18 kernel: [ 220.868163] User PSW : 0705200180000000 000003ffa223a5d4
Jun 20 08:39:18 kernel: [ 220.868165] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:1 AS:0 CC:2 PM:0 EA:3
Jun 20 08:39:18 kernel: [ 220.868165] User GPRS: 0000000000000000 000003ff0000004c 000003ffa234efc0 000003ffa2352370
Jun 20 08:39:18 kernel: [ 220.868166] 000003ffa2352352 2e302e313961392c 000003ffd2e7de48 000003ffa234efc2
Jun 20 08:39:18 kernel: [ 220.868167] 0000000000000010 0000000000000012 000003ff00004c49 2c302e302e313961
Jun 20 08:39:18 kernel: [ 220.868168] 000003ffa2389000 000003ffd2e7ed38 000003ffa2278eaa 000003ffd2e7dad0
Jun 20 08:39:18 kernel: [ 220.868177] User Code: 000003ffa223a5c6: 41d0d008 la %r13,8(%r13)
Jun 20 08:39:18 kernel: [ 220.868177] 000003ffa223a5ca: e3b0d0000002 ltg %r11,0(%r13)
Jun 20 08:39:18 kernel: [ 220.868177] #000003ffa223a5d0: a784ffe2 brc 8,3ffa223a594
Jun 20 08:39:18 kernel: [ 220.868177] >000003ffa223a5d4: e310b0000095 llh %r1,0(%r11)
Jun 20 08:39:18 kernel: [ 220.868177] 000003ffa223a5da: eca1fff66076 crj %r10,%r1,6,3ffa223a5c6
Jun 20 08:39:18 kernel: [ 220.868177] 000003ffa223a5e0: b9040048 lgr %r4,%r8
Jun 20 08:39:18 kernel: [ 220.868177] 000003ffa223a5e4: b9040037 lgr %r3,%r7
Jun 20 08:39:18 kernel: [ 220.868177] 000003ffa223a5e8: 4120b002 la %r2,2(%r11)
Jun 20 08:39:18 kernel: [ 220.868187] Last Breaking-Event-Address:
Jun 20 08:39:18 kernel: [ 220.868189] [<000003ffa223a5c2>] 0x3ffa223a5c2
Jun 20 08:39:23 main-menu[346]: INFO: Modifying debconf priority limit from 'high' to 'medium'
Jun 20 08:39:23 debconf: Setting debconf/priority to medium

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Can you try to use cio_ignore during installation, and does that mitigate this bug?

Specifically, can you use cio_ignore such that maximum number of visible FCP devices is no more than 100?

Could you please notify Hendrik Brueckner about this bug report?

From zfcp-config.c:

/* Debconf related definitions */
#define TEMPLATE_PREFIX "s390-zfcp/"
#define DEBCONF_CAPS "backup"
#define PRESEED_DELIM ","
#define SCSI_ASYNC_TRIES 15
#define SCSI_ASYNC_TIMEOUT 250000 /* useconds */
#define MAX_HOST_LIST_ITEMS 100
#define MAX_HOST_LIST_SIZE (MAX_HOST_LIST_ITEMS * (9 + 20 +1))
#define MAX_LUN_LIST_SIZE 2048

Revision history for this message
bugproxy (bugproxy) wrote : /var/log/installer

Default Comment by Bridge

tags: added: targetmilestone-inin16041
removed: targetmilestone-inin---
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2016-06-29 10:48 EDT-------
Limiting the number of available devices to those that are required (<100) would be a work around. I work on patch to dynamically allocate memory for detected FCP devices. This would then be equivalent to those patches in s390-netdevice and s390-dasd.

Thanks and kind regards,
Hendrik

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → Triaged
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-07-04 06:08 EDT-------
Patch solution has been submitted to Debian bug tracker:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=829562

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
importance: Undecided → Medium
Changed in s390-zfcp (Ubuntu):
status: New → In Progress
Changed in s390-zfcp (Ubuntu Xenial):
status: New → Triaged
importance: Undecided → Wishlist
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-zfcp - 1.0.4ubuntu1

---------------
s390-zfcp (1.0.4ubuntu1) yakkety; urgency=low

  * Merge from Debian unstable LP: #1596968. Remaining changes:
    - Run update-dev, to trigger device addition and hence loading of the
    zfcp module. Previously this was only loaded after hw/disk-detect
    which is too late, as this package step is already done at that point.

s390-zfcp (1.0.4) unstable; urgency=medium

  [ Hendrik Brueckner ]
  * Improve displaying numereous detected FCP devices by replacing the
    hard-coded limit with a dynamically allocated solution (Closes: #829562)

s390-zfcp (1.0.3) unstable; urgency=medium

  * Drop useless debian/po directory. Closes: #817206, #821173

 -- Dimitri John Ledkov <email address hidden> Mon, 25 Jul 2016 17:13:55 +0000

Changed in s390-zfcp (Ubuntu):
status: In Progress → Fix Released
Changed in s390-zfcp (Ubuntu Xenial):
assignee: nobody → Dimitri John Ledkov (xnox)
milestone: none → ubuntu-16.04.2
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-12-14 08:22 EDT-------
Changed target to 16.04.2 after discussion with Canonical

tags: added: targetmilestone-inin16042
removed: targetmilestone-inin16041
Changed in s390-zfcp (Ubuntu Xenial):
status: Triaged → Fix Committed
description: updated
Changed in s390-zfcp (Ubuntu Xenial):
status: Fix Committed → In Progress
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted s390-zfcp into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/s390-zfcp/1.0.2ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in s390-zfcp (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Please perform installation using "apt-setup/proposed=true" in the param file and confirm that installation on an LPAR with 100+ zfcp devices visible is now possible.

I have performed installation on our LPARs without regressions using 1.0.2ubuntu1.1 s390-zfcp package from proposed. However, the mainframe I am performing this test on has insufficient amount of zfcp devices to trigger the originally reported bug.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for s390-zfcp has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-zfcp - 1.0.2ubuntu1.1

---------------
s390-zfcp (1.0.2ubuntu1.1) xenial; urgency=medium

  [ Hendrik Brueckner ]
  * Improve displaying numereous detected FCP devices by replacing the
    hard-coded limit with a dynamically allocated solution LP: #1596968

 -- Dimitri John Ledkov <email address hidden> Wed, 14 Dec 2016 14:03:20 +0000

Changed in s390-zfcp (Ubuntu Xenial):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.