wsrep_flow_control_sent/recv not a global counter?

Bug #1180792 reported by Jay Janssen
30
This bug affects 7 people
Affects Status Importance Assigned to Milestone
Galera
Status tracked in 3.x
2.x
Won't Fix
Undecided
Unassigned
3.x
Fix Released
Wishlist
Alex Yurchenko
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC
5.5
Won't Fix
Undecided
Unassigned
5.6
Fix Released
Undecided
Unassigned

Bug Description

I execute this on one node to trigger FC:

mysql> flush tables with read lock;
Query OK, 0 rows affected (0.00 sec)

I can see a flow control message being sent and the cluster enters the pause state

root@perconadbt ~]# myq_status wsrep
Wsrep Cluster Node Queue Ops Bytes Flow Conflct
    time name P cnf # name cmt sta Up Dn Up Dn Up Dn pau snt dst lcf bfa
10:15:11 myclu P 15 3 node3 Sync T/T 0 0 0 38k 0 53M 0.0 0 196 0 0
10:15:13 myclu P 15 3 node3 Sync T/T 0 0 0 63 0 86K 0.0 0 202 0 0
10:15:14 myclu P 15 3 node3 Sync T/T 0 0 0 84 0 119K 0.0 0 187 0 0
10:15:15 myclu P 15 3 node3 Sync T/T 0 0 0 60 0 85K 0.0 0 191 0 0
10:15:16 myclu P 15 3 node3 Sync T/T 0 0 0 56 0 83K 0.0 0 199 0 0
10:15:17 myclu P 15 3 node3 Sync T/T 0 0 0 64 0 93K 0.0 0 192 0 0
10:15:18 myclu P 15 3 node3 Sync T/T 0 0 0 116 0 163K 0.0 0 186 0 0
10:15:19 myclu P 15 3 node3 Sync T/T 0 0 0 76 0 109K 0.0 0 179 0 0
10:15:20 myclu P 15 3 node3 Sync T/T 0 0 0 81 0 115K 0.0 0 184 0 0
10:15:21 myclu P 15 3 node3 Sync T/T 0 29 0 22 0 31K 0.1 1 190 0 0
10:15:23 myclu P 15 3 node3 Sync T/T 0 29 0 0 0 0 1.0 0 190 0 0
10:15:24 myclu P 15 3 node3 Sync T/T 0 29 0 0 0 0 1.0 0 190 0 0
10:15:25 myclu P 15 3 node3 Sync T/T 0 29 0 0 0 0 1.0 0 190 0 0
10:15:26 myclu P 15 3 node3 Sync T/T 0 29 0 0 0 0 1.0 0 190 0 0
10:15:27 myclu P 15 3 node3 Sync T/T 0 29 0 0 0 0 1.0 0 190 0 0

But, on that same node:

mysql> show global status like 'wsrep_flow_control%';
+---------------------------+----------+
| Variable_name | Value |
+---------------------------+----------+
| wsrep_flow_control_paused | 1.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
+---------------------------+----------+
3 rows in set (0.00 sec)

Why is flow_control_sent 0 here?

I also noted flow_control_recv is 0 on other nodes.

I thought these were global counters?

Related branches

Revision history for this message
Alex Yurchenko (ayurchen) wrote :

Jay, at the momemtn these are not monotonically increasing:

> 10:15:20 myclu P 15 3 node3 Sync T/T 0 0 0 81 0 115K 0.0 0 184 0 0
> 10:15:21 myclu P 15 3 node3 Sync T/T 0 29 0 22 0 31K 0.1 1 190 0 0
> 10:15:23 myclu P 15 3 node3 Sync T/T 0 29 0 0 0 0 1.0 0 190 0 0

Basically you will (almost) always have sent=0 when paused=1.0

Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

Yeah, I see

    stats->fc_sent = conn->stats_fc_sent; conn->stats_fc_sent = 0;

in gcs_get_stats.

May be they can be made into a cumulative counter? (if it helps).

Revision history for this message
Alex Yurchenko (ayurchen) wrote :

Yes, we're thinking of making them into cumulative counters. Most definitely they will be in 3.0. However this requires changing of wsrep API (addition of a flush status call), so...

Revision history for this message
Jay Janssen (jay-janssen) wrote : Re: [Bug 1180792] wsrep_flow_control_sent/recv not a global counter?

On May 16, 2013, at 12:35 PM, Raghavendra D Prabhu <email address hidden> wrote:

> May be they can be made into a cumulative counter? (if it helps).

Yes, I don't think it's useful if we can't see what nodes are historically sending fc messages.

Jay Janssen, MySQL Consulting Lead, Percona
http://about.me/jay.janssen

Revision history for this message
Jay Janssen (jay-janssen) wrote : Re: [Bug 1180792] Re: wsrep_flow_control_sent/recv not a global counter?

On May 16, 2013, at 12:57 PM, Alex Yurchenko <email address hidden> wrote:

> Yes, we're thinking of making them into cumulative counters. Most
> definitely they will be in 3.0. However this requires changing of wsrep
> API (addition of a flush status call), so...

Remember our discussion in Santa Clara…. PXC 5.5 will exist for many years at this point. Unless you backport 3.x to 5.5 we're going to be stuck with certain things for a very long time. I guess we can argue about what is and is not worth fixing in 2.x...

Jay Janssen, MySQL Consulting Lead, Percona
http://about.me/jay.janssen

Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

Added galera project to track for it.

Revision history for this message
Alex Yurchenko (ayurchen) wrote :

Jay, it is not 2.x vs. 3.x, it is API version: 23 vs. 24. Porting to 24 is not a big deal since it is mostly addition of new funtions, it just needs to get finalized good.

Changed in galera:
importance: Undecided → Wishlist
status: New → Confirmed
Revision history for this message
Alex Yurchenko (ayurchen) wrote :

Not fixing it in 2.x for backward compatibility reasons.

Changed in galera:
assignee: nobody → Alex Yurchenko (ayurchen)
milestone: none → 25.2.8
status: Confirmed → In Progress
Revision history for this message
Alex Yurchenko (ayurchen) wrote :
no longer affects: percona-xtradb-cluster
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.