Comment 29 for bug 1414818

------- Comment From <email address hidden> 2015-07-16 16:07 EDT-------
here is the verification
Perftest:
root@powerio-le13:~# uname -a
Linux powerio-le13 3.19.0-23-generic #24~14.04.1-Ubuntu SMP Wed Jul 8 11:17:19 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
root@powerio-le13:~# dpkg --list | grep perftest
ii perftest 1.2-OFED-1.4.2-2ubuntu1 ppc64el Infiniband verbs performance tests
root@powerio-le13:~# ib_send_bw 10.10.10.10
------------------------------------------------------------------
Send BW Test
Connection type : RC
Inline data is used up to 400 bytes message
local address: LID 0x02, QPN 0x002d, PSN 0x20dbd6
remote address: LID 0x02, QPN 0x002c, PSN 0xc3a9bf
Mtu : 2048
------------------------------------------------------------------
#bytes #iterations BW peak[MB/sec] BW average[MB/sec]
Warning: measured timestamp frequency 511.325 differs from nominal 3458 MHz
65536 1000 10695.39 10600.86
------------------------------------------------------------------

So perftest package is available and running.

Opensm
Having issues to find it
apt-get install opensm/trusty-proposed
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Release 'trusty-proposed' for 'opensm' was not found

ibutils
root@powerio-le13:~# dpkg --list | grep ibutils
ii ibutils 1.5.7-1ubuntu2 ppc64el InfiniBand network utilities
root@powerio-le13:~# ibdiagnet
Loading IBDIAGNET from: /usr/lib/powerpc64le-linux-gnu/ibdiagnet1.5.7
-W- Topology file is not specified.
Reports regarding cluster links will use direct routes.
Loading IBDM from: /usr/lib/powerpc64le-linux-gnu/ibdm1.5.7
-W- A few ports of local device are up.
Since port-num was not specified (-p option), port 1 of device 1 will be
used as the local port.
-I- Discovering ... 2 nodes (0 Switches & 2 CA-s) discovered.

-I---------------------------------------------------
-I- Bad Guids/LIDs Info
-I---------------------------------------------------
-I- No bad Guids were found

-I---------------------------------------------------
-I- Links With Logical State = INIT
-I---------------------------------------------------
-I- No bad Links (with logical state = INIT) were found

-I---------------------------------------------------
-I- General Device Info
-I---------------------------------------------------

-I---------------------------------------------------
-I- PM Counters Info
-I---------------------------------------------------
-I- No illegal PM counters values were found

-I---------------------------------------------------
-I- Fabric Partitions Report (see ibdiagnet.pkey for a full hosts list)
-I---------------------------------------------------
-I- PKey:0x7fff Hosts:2 full:2 limited:0

-I---------------------------------------------------
-I- IPoIB Subnets Check
-I---------------------------------------------------
-I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:2048Byte rate:10Gbps SL:0x00
-W- Suboptimal rate for group. Lowest member rate:40Gbps > group-rate:10Gbps

-I---------------------------------------------------
-I- Bad Links Info
-I- No bad link were found
-I---------------------------------------------------
----------------------------------------------------------------
-I- Stages Status Report:
STAGE Errors Warnings
Bad GUIDs/LIDs Check 0 0
Link State Active Check 0 0
General Devices Info Report 0 0
Performance Counters Report 0 0
Partitions Check 0 0
IPoIB Subnets Check 0 1

Please see /tmp/ibdiagnet.log for complete log
----------------------------------------------------------------

-I- Done. Run time was 1 seconds.
root@powerio-le13:~#

So I can find it and I can run test.

mstflint
with Mellanox CX2 works but not with other adapters
mstflint -d mlx4_0 q
Image type: ConnectX
FW Version: 2.9.1326
Device ID: 26448
Chip Revision: B0
Description: Port1 Port2
MACs: f45214cec124 f45214cec125
Board ID: (IBM0F30001010)
VSD:
PSID: IBM0F30001010

Connect IB
dpkg --list | grep mstflint
ii mstflint 1.4-OFED-1.4.2-1ubuntu1 ppc64el Mellanox firmware burning application
root@powerio-le13:~# mstflint -d mlx5_0 q
Unable to parse device name mlx5_0
*** ERROR *** Can not open mlx5_0: Invalid argument MFE_CR_ERROR

with CX3 40G
mstflint -d mlx4_0 q
Warning: memory access to device mlx4_0 failed: Input/output error.
Warning: Fallback on IO: much slower, and unsafe if device in use.
*** ERROR *** Can not open mlx4_0: No such file or directory MFE_CR_ERROR

dpkg --list | grep rds-tools
ii rds-tools 1.4.1-OFED-1.4.2-1ubuntu1 ppc64el Tools for the Infiniband RDS protocol
I can not find a partner to run rds ping but at least rds-info gives me output:

rds-info

RDS IB Connections:
LocalAddr RemoteAddr LocalDev RemoteDev

TCP Connections:
LocalAddr LPort RemoteAddr RPort HdrRemain DataRemain SentNxt ExpectUna SeenUna

Counters:
CounterName Value
conn_reset 0
recv_drop_bad_checksum 0
recv_drop_old_seq 0
recv_drop_no_sock 0
recv_drop_dead_sock 0
recv_deliver_raced 0
recv_delivered 0
recv_queued 0
recv_immediate_retry 0
recv_delayed_retry 0
recv_ack_required 0
recv_rdma_bytes 0
recv_ping 0
send_queue_empty 0
send_queue_full 0
send_lock_contention 0
send_lock_queue_raced 0
send_immediate_retry 0
send_delayed_retry 0
send_drop_acked 0
send_ack_required 0
send_queued 0
send_rdma 0
send_rdma_bytes 0
send_pong 0
page_remainder_hit 0
page_remainder_miss 0
copy_to_user 0
copy_from_user 0
cong_update_queued 0
cong_update_received 0
cong_send_error 0
cong_send_blocked 0
ib_connect_raced 0
ib_listen_closed_stale 0
ib_tx_cq_call 0
ib_tx_cq_event 0
ib_tx_ring_full 0
ib_tx_throttle 0
ib_tx_sg_mapping_failure 0
ib_tx_stalled 0
ib_tx_credit_updates 0
ib_rx_cq_call 0
ib_rx_cq_event 0
ib_rx_ring_empty 0
ib_rx_refill_from_cq 0
ib_rx_refill_from_thread 0
ib_rx_alloc_limit 0
ib_rx_credit_updates 0
ib_ack_sent 0
ib_ack_send_failure 0
ib_ack_send_delayed 0
ib_ack_send_piggybacked 0
ib_ack_received 0
ib_rdma_mr_alloc 0
ib_rdma_mr_free 0
ib_rdma_mr_used 0
ib_rdma_mr_pool_flush 0
ib_rdma_mr_pool_wait 0
ib_rdma_mr_pool_depleted 0
ib_atomic_cswp 0
ib_atomic_fadd 0
iw_connect_raced 0
iw_listen_closed_stale 0
iw_tx_cq_call 0
iw_tx_cq_event 0
iw_tx_ring_full 0
iw_tx_throttle 0
iw_tx_sg_mapping_failure 0
iw_tx_stalled 0
iw_tx_credit_updates 0
iw_rx_cq_call 0
iw_rx_cq_event 0
iw_rx_ring_empty 0
iw_rx_refill_from_cq 0
iw_rx_refill_from_thread 0
iw_rx_alloc_limit 0
iw_rx_credit_updates 0
iw_ack_sent 0
iw_ack_send_failure 0
iw_ack_send_delayed 0
iw_ack_send_piggybacked 0
iw_ack_received 0
iw_rdma_mr_alloc 0
iw_rdma_mr_free 0
iw_rdma_mr_used 0
iw_rdma_mr_pool_flush 0
iw_rdma_mr_pool_wait 0
iw_rdma_mr_pool_depleted 0
tcp_data_ready_calls 0
tcp_write_space_calls 0
tcp_sndbuf_full 0
tcp_connect_raced 0
tcp_listen_closed_stale 0

RDS Sockets:
BoundAddr BPort ConnAddr CPort SndBuf RcvBuf Inode
0.0.0.0 0 0.0.0.0 0 114688 114688 15499

RDS Connections:
LocalAddr RemoteAddr NextTX NextRX Flg

Receive Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes

Send Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes

Retransmit Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes

So in summary missing opensm and infiniband-diags. and mstflint is not working for some cards. I need to double check if the cards that I see falining here if they fail in Ubuntu 15.04.