Merge ~lucaskanashiro/ubuntu/+source/pacemaker:lp1896223-focal into ubuntu/+source/pacemaker:ubuntu/focal-devel

[Lucas Kanashiro]

Backport the no-quorum-policy=demote and on-fail=demote features for MSSQL servers. It was previously proposed by Rafael and reviewed by Christian here:

https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/pacemaker/+git/pacemaker/+merge/392511

There was a security update in the meantime and I needed to rebase the changes.

I still want to run the regression tests before moving forward but a review for the packaging work is appreciated :)

[Christian Ehrhardt  (paelzer)]

The content of the MP has changed a lot (some patch headers, a whitespace change).
Seems like a re-export from git with other git options after a rebase, but mostly nothing important - yet a lot to re-digest :-)

---

There are also other changes like crm_config_err -> pcmk__config_err, action_t -> pe_action_t, and so on

I re-reviewed these compared to the changes that formerly went in.
The patches still apply and didn't semantically change (mostly varibable names and such context updates).

Well, the code is complex and "applying" isn't everything.
Could be all good, or we have just lost all that we formerly had identified on backport.
Because the changes closely match the former backport notes.

Let me ask one question as an example - and depending on the outcome I'd ask to regenerate ALL patches with that in mind - or I'm happy as is and we can go on.
File: lp1896223-02-ef246ff-Fix-scheduler-disallow-on-fail-stop-for-stop.patch
Rafael: crm_config_err
Lucas: pcmk__config_err
But Lucas File still says at the top:
  7 [Backport]
  8
  9 This pacemaker version did not use pcmk__config_err() function for
 10 configuration warnings. It used crm_config_err() still.
So either update all the patch headers to have backport-notes that match the content.
OR I've found changes that got unintentionally changed - then please fix these.

I have checked the example above for pcmk__config_err, but it isn't touched in the patches that went in since the last merge by Rafel. So one of you must be wrong I guess?

---

One more thing that remains as a todo:
"One bit, on SRUs I usually proof-read the SRU details on the bugs description before we go to the SRU team to be denied. There is nothing there yet, please make sure to add it before uploading."

That I asked on the old MP and I'd still ask for it - yet your SRU experience is good and you won't make a bad bug description (so I don't need to review it) - just keep in mind that you need to be complete before uploading.

---

TL;DR:
- Please explain if the patch differences were intentional (or fix them up if not)
- please add SRU description before upload
- please complete the tests before upload

review: Needs Fixing

[Lucas Kanashiro (lucaskanashiro)]

You are right, I wrongly resolved the conflicts while I was cherry-picking the upstream commits. The Rafael's solution is the right one. I am going to redo everything to make sure the content of the patches are the same as Rafael's patches.

About the extra/missing lines in the patch headers might be the tooling I use to manage patches, I use git-buildpackage patch queue. I do not think this is a big deal.

The SRU bug description and the regression tests are the next steps, they are already in my todo list.

[Christian Ehrhardt  (paelzer)]

Now things LGTM - thanks.
The need for SRU-template addition is known.

+1 for this once the tests have completed.

review: Approve

[Lucas Kanashiro (lucaskanashiro)]

I imported the version 2.0.3-3ubuntu4.2 to this branch which was released after I was working on this. The only change I made was setting the version to 2.0.3-3ubuntu4.3, all the rest is the same. So I am still considering it as approved based on the last Christian's comment.

FWIW MS folks tested the package and said it is working as they were expecting. I'll be uploading the package.

[Lucas Kanashiro (lucaskanashiro)]

Uploaded:

$ git push pkg upload/2.0.3-3ubuntu4.3
Enumerating objects: 29, done.
Counting objects: 100% (29/29), done.
Delta compression using up to 32 threads
Compressing objects: 100% (24/24), done.
Writing objects: 100% (24/24), 19.60 KiB | 1.78 MiB/s, done.
Total 24 (delta 8), reused 0 (delta 0)
To ssh://git.launchpad.net/ubuntu/+source/pacemaker
 * [new tag] upload/2.0.3-3ubuntu4.3 -> upload/2.0.3-3ubuntu4.3
$ dput ubuntu ../pacemaker_2.0.3-3ubuntu4.3_source.changes
Checking signature on .changes
gpg: ../pacemaker_2.0.3-3ubuntu4.3_source.changes: Valid signature from F823A2729883C97C
Checking signature on .dsc
gpg: ../pacemaker_2.0.3-3ubuntu4.3.dsc: Valid signature from F823A2729883C97C
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading pacemaker_2.0.3-3ubuntu4.3.dsc: done.
  Uploading pacemaker_2.0.3-3ubuntu4.3.debian.tar.xz: done.
  Uploading pacemaker_2.0.3-3ubuntu4.3_source.changes: done.
Successfully uploaded packages.

diff --git a/debian/changelog b/debian/changelog
index e7ec315..c4ffd80 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,24 @@
+pacemaker (2.0.3-3ubuntu4.3) focal; urgency=medium
+
+  [ Rafael David Tinoco ]
+  * Post 2.0.3 features: on-fail=demote & no-quorum-policy=demote
+    (LP: #1896223). Added debian/patches/ubuntu-2.0.3-demote/*:
+    - lp1896223-01-f1f71b3-Refactor-scheduler-functionize-comparing-on-fail.patch
+    - lp1896223-02-ef246ff-Fix-scheduler-disallow-on-fail-stop-for-stop.patch
+    - lp1896223-03-8dceba7-Refactor-scheduler-use-more-appropriate-types.patch
+    - lp1896223-04-a4d6a20-Low-libpacemaker-don-t-force-stop-when-skipping.patch
+    - lp1896223-05-98c3b64-Log-libpacemaker-check-for-re-promotes-specifically.patch
+    - lp1896223-06-2f1e2df-Feature-xml-add-on-fail-demote-option-to-resources.patch
+    - lp1896223-07-874f75e-Feature-scheduler-new-on-fail-demote-recovery-policy.patch
+    - lp1896223-08-7eec572-Build-libcrmcommon-bump-CRM-feature-set.patch
+    - lp1896223-09-204961e-Doc-Pacemaker-Explained-document-new-on-fail.patch
+    - lp1896223-10-015b5c0-Doc-Pacemaker-Explained-document-no-quorum.patch
+    - lp1896223-11-0b68344-Refactor-scheduler-functionize-checking-quorum.patch
+    - lp1896223-12-b1ae359-Feature-scheduler-support-demote-choice-for.patch
+    - lp1896223-13-d4b9117-Doc-Pacemaker-Explained-correct-on-fail-default.patch
+
+ -- Lucas Kanashiro <[e-mail]>  Wed, 09 Dec 2020 10:27:00 -0300
+
 pacemaker (2.0.3-3ubuntu4.2) focal; urgency=medium
 
   * d/rules: Rebuild with QB_KILL_ATTRIBUTE_SECTION to overcome a problem in
diff --git a/debian/patches/series b/debian/patches/series
index c02030b..944751a 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -32,3 +32,19 @@ CVE-2020-25654-4.patch
 CVE-2020-25654-5.patch
 CVE-2020-25654-6.patch
 CVE-2020-25654-7.patch
+#
+# https://bugs.launchpad.net/bugs/1896223
+#
+ubuntu-2.0.3-demote/lp1896223-01-f1f71b3-Refactor-scheduler-functionize-comparing-on-fail.patch
+ubuntu-2.0.3-demote/lp1896223-02-ef246ff-Fix-scheduler-disallow-on-fail-stop-for-stop.patch
+ubuntu-2.0.3-demote/lp1896223-03-8dceba7-Refactor-scheduler-use-more-appropriate-types.patch
+ubuntu-2.0.3-demote/lp1896223-04-a4d6a20-Low-libpacemaker-don-t-force-stop-when-skipping.patch
+ubuntu-2.0.3-demote/lp1896223-05-98c3b64-Log-libpacemaker-check-for-re-promotes-specifically.patch
+ubuntu-2.0.3-demote/lp1896223-06-2f1e2df-Feature-xml-add-on-fail-demote-option-to-resources.patch
+ubuntu-2.0.3-demote/lp1896223-07-874f75e-Feature-scheduler-new-on-fail-demote-recovery-policy.patch
+ubuntu-2.0.3-demote/lp1896223-08-7eec572-Build-libcrmcommon-bump-CRM-feature-set.patch
+ubuntu-2.0.3-demote/lp1896223-09-204961e-Doc-Pacemaker-Explained-document-new-on-fail.patch
+ubuntu-2.0.3-demote/lp1896223-10-015b5c0-Doc-Pacemaker-Explained-document-no-quorum.patch
+ubuntu-2.0.3-demote/lp1896223-11-0b68344-Refactor-scheduler-functionize-checking-quorum.patch
+ubuntu-2.0.3-demote/lp1896223-12-b1ae359-Feature-scheduler-support-demote-choice-for.patch
+ubuntu-2.0.3-demote/lp1896223-13-d4b9117-Doc-Pacemaker-Explained-correct-on-fail-default.patch
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-01-f1f71b3-Refactor-scheduler-functionize-comparing-on-fail.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-01-f1f71b3-Refactor-scheduler-functionize-comparing-on-fail.patch
 new file mode 100644
index 0000000..796101c
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-01-f1f71b3-Refactor-scheduler-functionize-comparing-on-fail.patch
@@ -0,0 +1,181 @@
+From: Ken Gaillot <[e-mail]>
+Date: Thu, 28 May 2020 08:22:00 -0500
+Subject: Refactor: scheduler: functionize comparing on-fail values
+
+The action_fail_response enum values used for the "on-fail" operation
+meta-attribute were initially intended to be in order of severity.
+However as new values were added, they were added to the end (out of severity
+order) to preserve API backward compatibility.
+
+This resulted in a convoluted comparison of values that will only get worse as
+more values are added.
+
+This commit adds a comparison function to isolate that complexity.
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/f1f71b3
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ include/crm/pengine/common.h | 32 ++++++++++++------
+ lib/pengine/unpack.c         | 80 +++++++++++++++++++++++++++++++++++++++++---
+ 2 files changed, 97 insertions(+), 15 deletions(-)
+
+diff --git a/include/crm/pengine/common.h b/include/crm/pengine/common.h
+index e497f9c..450206e 100644
+--- a/include/crm/pengine/common.h
++++ b/include/crm/pengine/common.h
+@@ -29,18 +29,29 @@ extern "C" {
+ extern gboolean was_processing_error;
+ extern gboolean was_processing_warning;
+ 
+-/* order is significant here
+- * items listed in order of accending severeness
+- * more severe actions take precedent over lower ones
++/* The order is (partially) significant here; the values from action_fail_ignore
++ * through action_fail_fence are in order of increasing severity.
++ *
++ * @COMPAT The values should be ordered and numbered per the "TODO" comments
++ *         below, so all values are in order of severity and there is room for
++ *         future additions, but that would break API compatibility.
++ * @TODO   For now, we just use a function to compare the values specially, but
++ *         at the next compatibility break, we should arrange things properly.
+  */
+ enum action_fail_response {
+-    action_fail_ignore,
+-    action_fail_recover,
+-    action_fail_migrate,        /* recover by moving it somewhere else */
+-    action_fail_block,
+-    action_fail_stop,
+-    action_fail_standby,
+-    action_fail_fence,
++    action_fail_ignore,     // @TODO = 10
++    // @TODO action_fail_demote = 20,
++    action_fail_recover,    // @TODO = 30
++    // @TODO action_fail_reset_remote = 40,
++    // @TODO action_fail_restart_container = 50,
++    action_fail_migrate,    // @TODO = 60
++    action_fail_block,      // @TODO = 70
++    action_fail_stop,       // @TODO = 80
++    action_fail_standby,    // @TODO = 90
++    action_fail_fence,      // @TODO = 100
++
++    // @COMPAT Values below here are out of order for API compatibility
++
+     action_fail_restart_container,
+ 
+     /* This is reserved for internal use for remote node connection resources.
+@@ -51,6 +62,7 @@ enum action_fail_response {
+      */
+     action_fail_reset_remote,
+ 
++    action_fail_demote,
+ };
+ 
+ /* the "done" action must be the "pre" action +1 */
+diff --git a/lib/pengine/unpack.c b/lib/pengine/unpack.c
+index d337758..514207e 100644
+--- a/lib/pengine/unpack.c
++++ b/lib/pengine/unpack.c
+@@ -2724,6 +2724,78 @@ last_change_str(xmlNode *xml_op)
+     return ((when_s && *when_s)? when_s : "unknown time");
+ }
+ 
++/*!
++ * \internal
++ * \brief Compare two on-fail values
++ *
++ * \param[in] first   One on-fail value to compare
++ * \param[in] second  The other on-fail value to compare
++ *
++ * \return A negative number if second is more severe than first, zero if they
++ *         are equal, or a positive number if first is more severe than second.
++ * \note This is only needed until the action_fail_response values can be
++ *       renumbered at the next API compatibility break.
++ */
++static int
++cmp_on_fail(enum action_fail_response first, enum action_fail_response second)
++{
++    switch (first) {
++        case action_fail_reset_remote:
++            switch (second) {
++                case action_fail_ignore:
++                case action_fail_recover:
++                    return 1;
++                case action_fail_reset_remote:
++                    return 0;
++                default:
++                    return -1;
++            }
++            break;
++
++        case action_fail_restart_container:
++            switch (second) {
++                case action_fail_ignore:
++                case action_fail_recover:
++                case action_fail_reset_remote:
++                    return 1;
++                case action_fail_restart_container:
++                    return 0;
++                default:
++                    return -1;
++            }
++            break;
++
++        default:
++            break;
++    }
++    switch (second) {
++        case action_fail_reset_remote:
++            switch (first) {
++                case action_fail_ignore:
++                case action_fail_recover:
++                    return -1;
++                default:
++                    return 1;
++            }
++            break;
++
++        case action_fail_restart_container:
++            switch (first) {
++                case action_fail_ignore:
++                case action_fail_recover:
++                case action_fail_reset_remote:
++                    return -1;
++                default:
++                    return 1;
++            }
++            break;
++
++        default:
++            break;
++    }
++    return first - second;
++}
++
+ static void
+ unpack_rsc_op_failure(resource_t * rsc, node_t * node, int rc, xmlNode * xml_op, xmlNode ** last_failure,
+                       enum action_fail_response * on_fail, pe_working_set_t * data_set)
+@@ -2783,10 +2855,7 @@ unpack_rsc_op_failure(resource_t * rsc, node_t * node, int rc, xmlNode * xml_op,
+     }
+ 
+     action = custom_action(rsc, strdup(key), task, NULL, TRUE, FALSE, data_set);
+-    if ((action->on_fail <= action_fail_fence && *on_fail < action->on_fail) ||
+-        (action->on_fail == action_fail_reset_remote && *on_fail <= action_fail_recover) ||
+-        (action->on_fail == action_fail_restart_container && *on_fail <= action_fail_recover) ||
+-        (*on_fail == action_fail_restart_container && action->on_fail >= action_fail_migrate)) {
++    if (cmp_on_fail(*on_fail, action->on_fail) < 0) {
+         pe_rsc_trace(rsc, "on-fail %s -> %s for %s (%s)", fail2text(*on_fail),
+                      fail2text(action->on_fail), action->uuid, key);
+         *on_fail = action->on_fail;
+@@ -3630,7 +3699,8 @@ unpack_rsc_op(pe_resource_t *rsc, pe_node_t *node, xmlNode *xml_op,
+ 
+                 record_failed_op(xml_op, node, rsc, data_set);
+ 
+-                if (failure_strategy == action_fail_restart_container && *on_fail <= action_fail_recover) {
++                if ((failure_strategy == action_fail_restart_container)
++                    && cmp_on_fail(*on_fail, action_fail_recover) <= 0) {
+                     *on_fail = failure_strategy;
+                 }
+ 
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-02-ef246ff-Fix-scheduler-disallow-on-fail-stop-for-stop.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-02-ef246ff-Fix-scheduler-disallow-on-fail-stop-for-stop.patch
 new file mode 100644
index 0000000..72c7abc
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-02-ef246ff-Fix-scheduler-disallow-on-fail-stop-for-stop.patch
@@ -0,0 +1,53 @@
+From: Ken Gaillot <[e-mail]>
+Date: Thu, 28 May 2020 08:27:47 -0500
+Subject: Fix: scheduler: disallow on-fail=stop for stop operations
+
+because it would loop infinitely as long as the stop continued to fail
+
+  [Backport]
+
+  This pacemaker version did not use pcmk__config_err() function for
+  configuration warnings. It used crm_config_err() still.
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+
+Author: Ken Gaillot <[e-mail]>
+Origin: backport, https://github.com/ClusterLabs/pacemaker/commit/ef246ff
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ lib/pengine/utils.c | 14 ++++++++++++--
+ 1 file changed, 12 insertions(+), 2 deletions(-)
+
+diff --git a/lib/pengine/utils.c b/lib/pengine/utils.c
+index 3fc072f..ad5a09b 100644
+--- a/lib/pengine/utils.c
++++ b/lib/pengine/utils.c
+@@ -666,14 +666,24 @@ custom_action(resource_t * rsc, char *key, const char *task,
+     return action;
+ }
+ 
++static bool
++valid_stop_on_fail(const char *value)
++{
++    return safe_str_neq(value, "standby")
++           && safe_str_neq(value, "stop");
++}
++
+ static const char *
+ unpack_operation_on_fail(action_t * action)
+ {
+ 
+     const char *value = g_hash_table_lookup(action->meta, XML_OP_ATTR_ON_FAIL);
+ 
+-    if (safe_str_eq(action->task, CRMD_ACTION_STOP) && safe_str_eq(value, "standby")) {
+-        crm_config_err("on-fail=standby is not allowed for stop actions: %s", action->rsc->id);
++    if (safe_str_eq(action->task, CRMD_ACTION_STOP)
++        && !valid_stop_on_fail(value)) {
++        crm_config_err("Resetting '" XML_OP_ATTR_ON_FAIL "' for %s stop "
++                "action to default value because '%s' is not "
++                "allowed for stop", action->rsc->id, value);
+         return NULL;
+     } else if (safe_str_eq(action->task, CRMD_ACTION_DEMOTE) && !value) {
+         /* demote on_fail defaults to master monitor value if present */
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-03-8dceba7-Refactor-scheduler-use-more-appropriate-types.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-03-8dceba7-Refactor-scheduler-use-more-appropriate-types.patch
 new file mode 100644
index 0000000..303d874
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-03-8dceba7-Refactor-scheduler-use-more-appropriate-types.patch
@@ -0,0 +1,43 @@
+From: Ken Gaillot <[e-mail]>
+Date: Thu, 28 May 2020 08:50:33 -0500
+Subject: Refactor: scheduler: use more appropriate types in a couple places
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/8dceba7
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ lib/pengine/unpack.c | 5 ++---
+ 1 file changed, 2 insertions(+), 3 deletions(-)
+
+diff --git a/lib/pengine/unpack.c b/lib/pengine/unpack.c
+index 514207e..8f1ac1b 100644
+--- a/lib/pengine/unpack.c
++++ b/lib/pengine/unpack.c
+@@ -2206,7 +2206,7 @@ unpack_lrm_rsc_state(node_t * node, xmlNode * rsc_entry, pe_working_set_t * data
+     xmlNode *rsc_op = NULL;
+     xmlNode *last_failure = NULL;
+ 
+-    enum action_fail_response on_fail = FALSE;
++    enum action_fail_response on_fail = action_fail_ignore;
+     enum rsc_role_e saved_role = RSC_ROLE_UNKNOWN;
+ 
+     crm_trace("[%s] Processing %s on %s",
+@@ -2237,7 +2237,6 @@ unpack_lrm_rsc_state(node_t * node, xmlNode * rsc_entry, pe_working_set_t * data
+ 
+     /* process operations */
+     saved_role = rsc->role;
+-    on_fail = action_fail_ignore;
+     rsc->role = RSC_ROLE_UNKNOWN;
+     sorted_op_list = g_list_sort(op_list, sort_op_by_callid);
+ 
+@@ -3331,7 +3330,7 @@ int pe__target_rc_from_xml(xmlNode *xml_op)
+ static enum action_fail_response
+ get_action_on_fail(resource_t *rsc, const char *key, const char *task, pe_working_set_t * data_set) 
+ {
+-    int result = action_fail_recover;
++    enum action_fail_response result = action_fail_recover;
+     action_t *action = custom_action(rsc, strdup(key), task, NULL, TRUE, FALSE, data_set);
+ 
+     result = action->on_fail;
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-04-a4d6a20-Low-libpacemaker-don-t-force-stop-when-skipping.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-04-a4d6a20-Low-libpacemaker-don-t-force-stop-when-skipping.patch
 new file mode 100644
index 0000000..baad36a
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-04-a4d6a20-Low-libpacemaker-don-t-force-stop-when-skipping.patch
@@ -0,0 +1,45 @@
+From: Ken Gaillot <[e-mail]>
+Date: Tue, 2 Jun 2020 12:05:57 -0500
+Subject: Low: libpacemaker: don't force stop when skipping reload of failed
+ resource
+
+Normal failure recovery will apply, which will stop if needed.
+
+(The stop was forced as of 2558d76f.)
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/a4d6a20
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ lib/pacemaker/pcmk_sched_native.c | 16 +++++++++++++---
+ 1 file changed, 13 insertions(+), 3 deletions(-)
+
+diff --git a/lib/pacemaker/pcmk_sched_native.c b/lib/pacemaker/pcmk_sched_native.c
+index bbf3eb7..04552c4 100644
+--- a/lib/pacemaker/pcmk_sched_native.c
++++ b/lib/pacemaker/pcmk_sched_native.c
+@@ -3270,9 +3270,19 @@ ReloadRsc(resource_t * rsc, node_t *node, pe_working_set_t * data_set)
+         pe_rsc_trace(rsc, "%s: unmanaged", rsc->id);
+         return;
+ 
+-    } else if (is_set(rsc->flags, pe_rsc_failed) || is_set(rsc->flags, pe_rsc_start_pending)) {
+-        pe_rsc_trace(rsc, "%s: general resource state: flags=0x%.16llx", rsc->id, rsc->flags);
+-        stop_action(rsc, node, FALSE); /* Force a full restart, overkill? */
++    } else if (is_set(rsc->flags, pe_rsc_failed)) {
++        /* We don't need to specify any particular actions here, normal failure
++         * recovery will apply.
++         */
++        pe_rsc_trace(rsc, "%s: preventing reload because failed", rsc->id);
++        return;
++
++    } else if (is_set(rsc->flags, pe_rsc_start_pending)) {
++        /* If a resource's configuration changed while a start was pending,
++         * force a full restart.
++         */
++        pe_rsc_trace(rsc, "%s: preventing reload because start pending", rsc->id);
++        stop_action(rsc, node, FALSE);
+         return;
+ 
+     } else if (node == NULL) {
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-05-98c3b64-Log-libpacemaker-check-for-re-promotes-specifically.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-05-98c3b64-Log-libpacemaker-check-for-re-promotes-specifically.patch
 new file mode 100644
index 0000000..7620a7c
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-05-98c3b64-Log-libpacemaker-check-for-re-promotes-specifically.patch
@@ -0,0 +1,46 @@
+From: Ken Gaillot <[e-mail]>
+Date: Mon, 13 Apr 2020 12:23:22 -0500
+Subject: Log: libpacemaker: check for re-promotes specifically
+
+If a promotable clone instance is being demoted and promoted on its current
+node, without also stopping and starting, it previously would be logged as
+"Leave" indicating unchanged, because the current and next role are the same.
+
+Now, check for this situation specifically, and log it as "Re-promote".
+
+Currently, the scheduler is not capable of generating this situation, but
+upcoming changes will.
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/98c3b64
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ lib/pacemaker/pcmk_sched_native.c | 12 ++++++++++--
+ 1 file changed, 10 insertions(+), 2 deletions(-)
+
+diff --git a/lib/pacemaker/pcmk_sched_native.c b/lib/pacemaker/pcmk_sched_native.c
+index 04552c4..7c193fa 100644
+--- a/lib/pacemaker/pcmk_sched_native.c
++++ b/lib/pacemaker/pcmk_sched_native.c
+@@ -2466,9 +2466,17 @@ LogActions(resource_t * rsc, pe_working_set_t * data_set, gboolean terminal)
+         } else if (is_set(rsc->flags, pe_rsc_reload)) {
+             LogAction("Reload", rsc, current, next, start, NULL, terminal);
+ 
++
+         } else if (start == NULL || is_set(start->flags, pe_action_optional)) {
+-            pe_rsc_info(rsc, "Leave   %s\t(%s %s)", rsc->id, role2text(rsc->role),
+-                        next->details->uname);
++            if ((demote != NULL) && (promote != NULL)
++                && is_not_set(demote->flags, pe_action_optional)
++                && is_not_set(promote->flags, pe_action_optional)) {
++                LogAction("Re-promote", rsc, current, next, promote, demote,
++                          terminal);
++            } else {
++                pe_rsc_info(rsc, "Leave   %s\t(%s %s)", rsc->id,
++                            role2text(rsc->role), next->details->uname);
++            }
+ 
+         } else if (start && is_set(start->flags, pe_action_runnable) == FALSE) {
+             LogAction("Stop", rsc, current, NULL, stop,
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-06-2f1e2df-Feature-xml-add-on-fail-demote-option-to-resources.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-06-2f1e2df-Feature-xml-add-on-fail-demote-option-to-resources.patch
 new file mode 100644
index 0000000..d6aeb97
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-06-2f1e2df-Feature-xml-add-on-fail-demote-option-to-resources.patch
@@ -0,0 +1,38 @@
+From: Ken Gaillot <[e-mail]>
+Date: Tue, 26 May 2020 17:50:48 -0500
+Subject: Feature: xml: add on-fail="demote" option to resources schema
+
+We don't need an XML schema version bump because it was already bumped since
+the last release, for the rsc_expression/op_expression feature.
+
+  [Backport]
+
+  Original patch changes xml/resources-3.4.rng. As we're backporting
+  features to Ubuntu 2.0.4 release, which only defines schema up to
+  xml/resources-3.2.rng, and using a new minor version only, for this
+  new Ubuntu only feature set (3.3.1), this patch adds the feature to
+  the 3.2 resources schema instead of a 3.4.
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+
+Author: Ken Gaillot <[e-mail]>
+Origin: backport, https://github.com/ClusterLabs/pacemaker/commit/2f1e2df
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ xml/resources-3.2.rng | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/xml/resources-3.2.rng b/xml/resources-3.2.rng
+index 44656d6..1930508 100644
+--- a/xml/resources-3.2.rng
++++ b/xml/resources-3.2.rng
+@@ -388,6 +388,7 @@
+                 <choice>
+                   <value>ignore</value>
+                   <value>block</value>
++                  <value>demote</value>
+                   <value>stop</value>
+                   <value>restart</value>
+                   <value>standby</value>
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-07-874f75e-Feature-scheduler-new-on-fail-demote-recovery-policy.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-07-874f75e-Feature-scheduler-new-on-fail-demote-recovery-policy.patch
 new file mode 100644
index 0000000..ae96daf
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-07-874f75e-Feature-scheduler-new-on-fail-demote-recovery-policy.patch
@@ -0,0 +1,355 @@
+From: Ken Gaillot <[e-mail]>
+Date: Thu, 28 May 2020 08:29:37 -0500
+Subject: Feature: scheduler: new on-fail="demote" recovery policy for
+ promoted resources
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/874f75e
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ include/crm/pengine/pe_types.h    |  1 +
+ lib/pacemaker/pcmk_sched_native.c | 25 +++++++++++++++----
+ lib/pengine/common.c              |  3 +++
+ lib/pengine/unpack.c              | 51 ++++++++++++++++++++++++++++++++++++---
+ lib/pengine/utils.c               | 34 ++++++++++++++++++++++----
+ 5 files changed, 101 insertions(+), 13 deletions(-)
+
+diff --git a/include/crm/pengine/pe_types.h b/include/crm/pengine/pe_types.h
+index 23e1c46..6e5cbcc 100644
+--- a/include/crm/pengine/pe_types.h
++++ b/include/crm/pengine/pe_types.h
+@@ -235,6 +235,7 @@ struct pe_node_s {
+ #  define pe_rsc_allocating                 0x00000200ULL
+ #  define pe_rsc_merging                    0x00000400ULL
+ 
++#  define pe_rsc_stop                       0x00001000ULL
+ #  define pe_rsc_reload                     0x00002000ULL
+ #  define pe_rsc_allow_remote_remotes       0x00004000ULL
+ 
+diff --git a/lib/pacemaker/pcmk_sched_native.c b/lib/pacemaker/pcmk_sched_native.c
+index 7c193fa..3ce75b8 100644
+--- a/lib/pacemaker/pcmk_sched_native.c
++++ b/lib/pacemaker/pcmk_sched_native.c
+@@ -1122,6 +1122,7 @@ native_create_actions(resource_t * rsc, pe_working_set_t * data_set)
+     node_t *chosen = NULL;
+     node_t *current = NULL;
+     gboolean need_stop = FALSE;
++    bool need_promote = FALSE;
+     gboolean is_moving = FALSE;
+     gboolean allow_migrate = is_set(rsc->flags, pe_rsc_allow_migrate) ? TRUE : FALSE;
+ 
+@@ -1226,8 +1227,15 @@ native_create_actions(resource_t * rsc, pe_working_set_t * data_set)
+         need_stop = TRUE;
+ 
+     } else if (is_set(rsc->flags, pe_rsc_failed)) {
+-        pe_rsc_trace(rsc, "Recovering %s", rsc->id);
+-        need_stop = TRUE;
++        if (is_set(rsc->flags, pe_rsc_stop)) {
++            need_stop = TRUE;
++            pe_rsc_trace(rsc, "Recovering %s", rsc->id);
++        } else {
++            pe_rsc_trace(rsc, "Recovering %s by demotion", rsc->id);
++            if (rsc->next_role == RSC_ROLE_MASTER) {
++                need_promote = TRUE;
++            }
++        }
+ 
+     } else if (is_set(rsc->flags, pe_rsc_block)) {
+         pe_rsc_trace(rsc, "Block %s", rsc->id);
+@@ -1261,10 +1269,16 @@ native_create_actions(resource_t * rsc, pe_working_set_t * data_set)
+ 
+ 
+     while (rsc->role <= rsc->next_role && role != rsc->role && is_not_set(rsc->flags, pe_rsc_block)) {
++        bool required = need_stop;
++
+         next_role = rsc_state_matrix[role][rsc->role];
++        if ((next_role == RSC_ROLE_MASTER) && need_promote) {
++            required = true;
++        }
+         pe_rsc_trace(rsc, "Up:   Executing: %s->%s (%s)%s", role2text(role), role2text(next_role),
+-                     rsc->id, need_stop ? " required" : "");
+-        if (rsc_action_matrix[role][next_role] (rsc, chosen, !need_stop, data_set) == FALSE) {
++                     rsc->id, (required? " required" : ""));
++        if (rsc_action_matrix[role][next_role](rsc, chosen, !required,
++                                               data_set) == FALSE) {
+             break;
+         }
+         role = next_role;
+@@ -2527,7 +2541,8 @@ LogActions(resource_t * rsc, pe_working_set_t * data_set, gboolean terminal)
+ 
+         free(key);
+ 
+-    } else if (stop && is_set(rsc->flags, pe_rsc_failed)) {
++    } else if (stop && is_set(rsc->flags, pe_rsc_failed)
++               && is_set(rsc->flags, pe_rsc_stop)) {
+         /* 'stop' may be NULL if the failure was ignored */
+         LogAction("Recover", rsc, current, next, stop, start, terminal);
+         STOP_SANITY_ASSERT(__LINE__);
+diff --git a/lib/pengine/common.c b/lib/pengine/common.c
+index da39c99..fcd7cf0 100644
+--- a/lib/pengine/common.c
++++ b/lib/pengine/common.c
+@@ -198,6 +198,9 @@ fail2text(enum action_fail_response fail)
+         case action_fail_ignore:
+             result = "ignore";
+             break;
++        case action_fail_demote:
++            result = "demote";
++            break;
+         case action_fail_block:
+             result = "block";
+             break;
+diff --git a/lib/pengine/unpack.c b/lib/pengine/unpack.c
+index 8f1ac1b..e690c4e 100644
+--- a/lib/pengine/unpack.c
++++ b/lib/pengine/unpack.c
+@@ -100,6 +100,7 @@ pe_fence_node(pe_working_set_t * data_set, node_t * node, const char *reason)
+                  */
+                 node->details->remote_requires_reset = TRUE;
+                 set_bit(rsc->flags, pe_rsc_failed);
++                set_bit(rsc->flags, pe_rsc_stop);
+             }
+         }
+ 
+@@ -109,6 +110,7 @@ pe_fence_node(pe_working_set_t * data_set, node_t * node, const char *reason)
+                  "and guest resource no longer exists",
+                  node->details->uname, reason);
+         set_bit(node->details->remote_rsc->flags, pe_rsc_failed);
++        set_bit(node->details->remote_rsc->flags, pe_rsc_stop);
+ 
+     } else if (pe__is_remote_node(node)) {
+         resource_t *rsc = node->details->remote_rsc;
+@@ -1898,6 +1900,7 @@ process_rsc_state(resource_t * rsc, node_t * node,
+          */
+         if (pe__is_guest_node(node)) {
+             set_bit(rsc->flags, pe_rsc_failed);
++            set_bit(rsc->flags, pe_rsc_stop);
+             should_fence = TRUE;
+ 
+         } else if (is_set(data_set->flags, pe_flag_stonith_enabled)) {
+@@ -1940,6 +1943,11 @@ process_rsc_state(resource_t * rsc, node_t * node,
+             /* nothing to do */
+             break;
+ 
++        case action_fail_demote:
++            set_bit(rsc->flags, pe_rsc_failed);
++            demote_action(rsc, node, FALSE);
++            break;
++
+         case action_fail_fence:
+             /* treat it as if it is still running
+              * but also mark the node as unclean
+@@ -1976,12 +1984,14 @@ process_rsc_state(resource_t * rsc, node_t * node,
+         case action_fail_recover:
+             if (rsc->role != RSC_ROLE_STOPPED && rsc->role != RSC_ROLE_UNKNOWN) {
+                 set_bit(rsc->flags, pe_rsc_failed);
++                set_bit(rsc->flags, pe_rsc_stop);
+                 stop_action(rsc, node, FALSE);
+             }
+             break;
+ 
+         case action_fail_restart_container:
+             set_bit(rsc->flags, pe_rsc_failed);
++            set_bit(rsc->flags, pe_rsc_stop);
+ 
+             if (rsc->container && pe_rsc_is_bundled(rsc)) {
+                 /* A bundle's remote connection can run on a different node than
+@@ -2000,6 +2010,7 @@ process_rsc_state(resource_t * rsc, node_t * node,
+ 
+         case action_fail_reset_remote:
+             set_bit(rsc->flags, pe_rsc_failed);
++            set_bit(rsc->flags, pe_rsc_stop);
+             if (is_set(data_set->flags, pe_flag_stonith_enabled)) {
+                 tmpnode = NULL;
+                 if (rsc->is_remote_node) {
+@@ -2054,8 +2065,17 @@ process_rsc_state(resource_t * rsc, node_t * node,
+         }
+ 
+         native_add_running(rsc, node, data_set);
+-        if (on_fail != action_fail_ignore) {
+-            set_bit(rsc->flags, pe_rsc_failed);
++        switch (on_fail) {
++            case action_fail_ignore:
++                break;
++            case action_fail_demote:
++            case action_fail_block:
++                set_bit(rsc->flags, pe_rsc_failed);
++                break;
++            default:
++                set_bit(rsc->flags, pe_rsc_failed);
++                set_bit(rsc->flags, pe_rsc_stop);
++                break;
+         }
+ 
+     } else if (rsc->clone_name && strchr(rsc->clone_name, ':') != NULL) {
+@@ -2549,6 +2569,7 @@ unpack_migrate_to_success(pe_resource_t *rsc, pe_node_t *node, xmlNode *xml_op,
+         } else {
+             /* Consider it failed here - forces a restart, prevents migration */
+             set_bit(rsc->flags, pe_rsc_failed);
++            set_bit(rsc->flags, pe_rsc_stop);
+             clear_bit(rsc->flags, pe_rsc_allow_migrate);
+         }
+     }
+@@ -2739,9 +2760,21 @@ static int
+ cmp_on_fail(enum action_fail_response first, enum action_fail_response second)
+ {
+     switch (first) {
++        case action_fail_demote:
++            switch (second) {
++                case action_fail_ignore:
++                    return 1;
++                case action_fail_demote:
++                    return 0;
++                default:
++                    return -1;
++            }
++            break;
++
+         case action_fail_reset_remote:
+             switch (second) {
+                 case action_fail_ignore:
++                case action_fail_demote:
+                 case action_fail_recover:
+                     return 1;
+                 case action_fail_reset_remote:
+@@ -2754,6 +2787,7 @@ cmp_on_fail(enum action_fail_response first, enum action_fail_response second)
+         case action_fail_restart_container:
+             switch (second) {
+                 case action_fail_ignore:
++                case action_fail_demote:
+                 case action_fail_recover:
+                 case action_fail_reset_remote:
+                     return 1;
+@@ -2768,9 +2802,13 @@ cmp_on_fail(enum action_fail_response first, enum action_fail_response second)
+             break;
+     }
+     switch (second) {
++        case action_fail_demote:
++            return (first == action_fail_ignore)? -1 : 1;
++
+         case action_fail_reset_remote:
+             switch (first) {
+                 case action_fail_ignore:
++                case action_fail_demote:
+                 case action_fail_recover:
+                     return -1;
+                 default:
+@@ -2781,6 +2819,7 @@ cmp_on_fail(enum action_fail_response first, enum action_fail_response second)
+         case action_fail_restart_container:
+             switch (first) {
+                 case action_fail_ignore:
++                case action_fail_demote:
+                 case action_fail_recover:
+                 case action_fail_reset_remote:
+                     return -1;
+@@ -3381,7 +3420,11 @@ update_resource_state(resource_t * rsc, node_t * node, xmlNode * xml_op, const c
+         clear_past_failure = TRUE;
+ 
+     } else if (safe_str_eq(task, CRMD_ACTION_DEMOTE)) {
+-        /* Demote from Master does not clear an error */
++
++        if (*on_fail == action_fail_demote) {
++            // Demote clears an error only if on-fail=demote
++            clear_past_failure = TRUE;
++        }
+         rsc->role = RSC_ROLE_SLAVE;
+ 
+     } else if (safe_str_eq(task, CRMD_ACTION_MIGRATED)) {
+@@ -3409,6 +3452,7 @@ update_resource_state(resource_t * rsc, node_t * node, xmlNode * xml_op, const c
+ 
+             case action_fail_block:
+             case action_fail_ignore:
++            case action_fail_demote:
+             case action_fail_recover:
+             case action_fail_restart_container:
+                 *on_fail = action_fail_ignore;
+@@ -3669,6 +3713,7 @@ unpack_rsc_op(pe_resource_t *rsc, pe_node_t *node, xmlNode *xml_op,
+                  * that, ensure the remote connection is considered failed.
+                  */
+                 set_bit(node->details->remote_rsc->flags, pe_rsc_failed);
++                set_bit(node->details->remote_rsc->flags, pe_rsc_stop);
+             }
+ 
+             // fall through
+diff --git a/lib/pengine/utils.c b/lib/pengine/utils.c
+index ad5a09b..e57d858 100644
+--- a/lib/pengine/utils.c
++++ b/lib/pengine/utils.c
+@@ -670,6 +670,7 @@ static bool
+ valid_stop_on_fail(const char *value)
+ {
+     return safe_str_neq(value, "standby")
++           && safe_str_neq(value, "demote")
+            && safe_str_neq(value, "stop");
+ }
+ 
+@@ -677,6 +678,11 @@ static const char *
+ unpack_operation_on_fail(action_t * action)
+ {
+ 
++    const char *name = NULL;
++    const char *role = NULL;
++    const char *on_fail = NULL;
++    const char *interval_spec = NULL;
++    const char *enabled = NULL;
+     const char *value = g_hash_table_lookup(action->meta, XML_OP_ATTR_ON_FAIL);
+ 
+     if (safe_str_eq(action->task, CRMD_ACTION_STOP)
+@@ -685,14 +691,10 @@ unpack_operation_on_fail(action_t * action)
+                 "action to default value because '%s' is not "
+                 "allowed for stop", action->rsc->id, value);
+         return NULL;
++
+     } else if (safe_str_eq(action->task, CRMD_ACTION_DEMOTE) && !value) {
+         /* demote on_fail defaults to master monitor value if present */
+         xmlNode *operation = NULL;
+-        const char *name = NULL;
+-        const char *role = NULL;
+-        const char *on_fail = NULL;
+-        const char *interval_spec = NULL;
+-        const char *enabled = NULL;
+ 
+         CRM_CHECK(action->rsc != NULL, return NULL);
+ 
+@@ -715,10 +717,28 @@ unpack_operation_on_fail(action_t * action)
+                 continue;
+             } else if (crm_parse_interval_spec(interval_spec) == 0) {
+                 continue;
++            } else if (safe_str_eq(on_fail, "demote")) {
++                continue;
+             }
+ 
+             value = on_fail;
+         }
++    } else if (safe_str_eq(value, "demote")) {
++        name = crm_element_value(action->op_entry, "name");
++        role = crm_element_value(action->op_entry, "role");
++        on_fail = crm_element_value(action->op_entry, XML_OP_ATTR_ON_FAIL);
++        interval_spec = crm_element_value(action->op_entry,
++                                          XML_LRM_ATTR_INTERVAL);
++
++        if (safe_str_neq(name, CRMD_ACTION_PROMOTE)
++            && (safe_str_neq(name, CRMD_ACTION_STATUS)
++                || safe_str_neq(role, "Master")
++                || (crm_parse_interval_spec(interval_spec) == 0))) {
++            crm_config_err("Resetting '" XML_OP_ATTR_ON_FAIL "' for %s %s "
++                             "action to default value because 'demote' is not "
++                             "allowed for it", action->rsc->id, name);
++            return NULL;
++        }
+     }
+ 
+     return value;
+@@ -1097,6 +1117,10 @@ unpack_operation(action_t * action, xmlNode * xml_obj, resource_t * container,
+             value = NULL;
+         }
+ 
++    } else if (safe_str_eq(value, "demote")) {
++        action->on_fail = action_fail_demote;
++        value = "demote instance";
++
+     } else {
+         pe_err("Resource %s: Unknown failure type (%s)", action->rsc->id, value);
+         value = NULL;
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-08-7eec572-Build-libcrmcommon-bump-CRM-feature-set.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-08-7eec572-Build-libcrmcommon-bump-CRM-feature-set.patch
 new file mode 100644
index 0000000..95033ed
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-08-7eec572-Build-libcrmcommon-bump-CRM-feature-set.patch
@@ -0,0 +1,51 @@
+From: Ken Gaillot <[e-mail]>
+Date: Fri, 5 Jun 2020 10:02:05 -0500
+Subject: Build: libcrmcommon: bump CRM feature set
+
+... for op_expression/rsc_expression rules, on-fail=demote, and
+no-quorum-policy=demote
+
+  [Backport]
+
+  The features op_expression/rsc_expression are not included in this
+  Ubuntu pacemaker release. The features being backported are only
+  on-fail/no-quorum-policy=demote. For this reason, instead of using the
+  upstream feature set version (3.4.0), I'm using Ubuntu own feature set
+  version 3.2.1 (using a minor-minor version).
+
+  Note: I have done the same thing in Pacemaker for Ubuntu Groovy, but,
+  because that pacemaker had feature set version 3.3.0, I used version
+  3.3.1.
+
+  There is no problem in having 3.2.1 supporting the feature, together
+  with 3.3.1, as the minor-minor version serves for that purpose: to
+  backport features in pacemaker distribution versions. When a cluster
+  is upgraded from Focal (3.2.1) to Groovy (3.3.1), the feature will
+  exist there as well.
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+
+Author: Ken Gaillot <[e-mail]>
+Origin: backport, https://github.com/ClusterLabs/pacemaker/commit/7eec572
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ include/crm/crm.h | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/include/crm/crm.h b/include/crm/crm.h
+index cbf72d3..35928bb 100644
+--- a/include/crm/crm.h
++++ b/include/crm/crm.h
+@@ -50,8 +50,9 @@ extern "C" {
+  *           XML v2 patchsets are created by default
+  * >=3.0.13: Fail counts include operation name and interval
+  * >=3.2.0:  DC supports PCMK_LRM_OP_INVALID and PCMK_LRM_OP_NOT_CONNECTED
++ * >=3.2.1:  UBUNTU: on-fail=demote + no-quorum-policy=demote (3.4.0 backport)
+  */
+-#  define CRM_FEATURE_SET		"3.2.0"
++#  define CRM_FEATURE_SET		"3.2.1"
+ 
+ #  define EOS		'\0'
+ #  define DIMOF(a)	((int) (sizeof(a)/sizeof(a[0])) )
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-09-204961e-Doc-Pacemaker-Explained-document-new-on-fail.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-09-204961e-Doc-Pacemaker-Explained-document-new-on-fail.patch
 new file mode 100644
index 0000000..d993858
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-09-204961e-Doc-Pacemaker-Explained-document-new-on-fail.patch
@@ -0,0 +1,67 @@
+From: Ken Gaillot <[e-mail]>
+Date: Tue, 26 May 2020 18:04:32 -0500
+Subject: Doc: Pacemaker Explained: document new on-fail="demote" option
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/204961e
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ doc/Pacemaker_Explained/en-US/Ch-Resources.txt | 36 ++++++++++++++++++++++++++
+ 1 file changed, 36 insertions(+)
+
+diff --git a/doc/Pacemaker_Explained/en-US/Ch-Resources.txt b/doc/Pacemaker_Explained/en-US/Ch-Resources.txt
+index d8e7115..9df9243 100644
+--- a/doc/Pacemaker_Explained/en-US/Ch-Resources.txt
++++ b/doc/Pacemaker_Explained/en-US/Ch-Resources.txt
+@@ -676,6 +676,10 @@ a|The action to take if this action ever fails. Allowed values:
+ * +ignore:+ Pretend the resource did not fail.
+ * +block:+ Don't perform any further operations on the resource.
+ * +stop:+ Stop the resource and do not start it elsewhere.
++* +demote:+ Demote the resource, without a full restart. This is valid only for
++  +promote+ actions, and for +monitor+ actions with both a nonzero +interval+
++  and +role+ set to +Master+; for any other action, a configuration error will
++  be logged, and the default behavior will be used.
+ * +restart:+ Stop the resource and start it again (possibly on a different node).
+ * +fence:+ STONITH the node on which the resource failed.
+ * +standby:+ Move _all_ resources away from the node on which the resource failed.
+@@ -714,6 +718,38 @@ indexterm:[Action,Property,on-fail]
+ 
+ |=========================================================
+ 
++[NOTE]
++====
++When +on-fail+ is set to +demote+, recovery from failure by a successful demote
++causes the cluster to recalculate whether and where a new instance should be
++promoted. The node with the failure is eligible, so if master scores have not
++changed, it will be promoted again.
++
++There is no direct equivalent of +migration-threshold+ for the master role, but
++the same effect can be achieved with a location constraint using a
++<<ch-rules,rule>> with a node attribute expression for the resource's fail
++count.
++
++For example, to immediately ban the master role from a node with any failed
++promote or master monitor:
++[source,XML]
++----
++<rsc_location id="loc1" rsc="my_primitive">
++    <rule id="rule1" score="-INFINITY" role="Master" boolean-op="or">
++      <expression id="expr1" attribute="fail-count-my_primitive#promote_0"
++        operation="gte" value="1"/>
++      <expression id="expr2" attribute="fail-count-my_primitive#monitor_10000"
++        operation="gte" value="1"/>
++    </rule>
++</rsc_location>
++----
++
++This example assumes that there is a promotable clone of the +my_primitive+
++resource (note that the primitive name, not the clone name, is used in the
++rule), and that there is a recurring 10-second-interval monitor configured for
++the master role (fail count attributes specify the interval in milliseconds).
++====
++
+ [[s-resource-monitoring]]
+ === Monitoring Resources for Failure ===
+ 
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-10-015b5c0-Doc-Pacemaker-Explained-document-no-quorum.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-10-015b5c0-Doc-Pacemaker-Explained-document-no-quorum.patch
 new file mode 100644
index 0000000..2573479
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-10-015b5c0-Doc-Pacemaker-Explained-document-no-quorum.patch
@@ -0,0 +1,26 @@
+From: Ken Gaillot <[e-mail]>
+Date: Thu, 28 May 2020 12:13:20 -0500
+Subject: Doc: Pacemaker Explained: document no-quorum-policy=demote
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/015b5c0
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ doc/Pacemaker_Explained/en-US/Ch-Options.txt | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/doc/Pacemaker_Explained/en-US/Ch-Options.txt b/doc/Pacemaker_Explained/en-US/Ch-Options.txt
+index f864987..d344ecd 100644
+--- a/doc/Pacemaker_Explained/en-US/Ch-Options.txt
++++ b/doc/Pacemaker_Explained/en-US/Ch-Options.txt
+@@ -181,6 +181,8 @@ What to do when the cluster does not have quorum.  Allowed values:
+ * +ignore:+ continue all resource management
+ * +freeze:+ continue resource management, but don't recover resources from nodes not in the affected partition
+ * +stop:+ stop all resources in the affected cluster partition
++* +demote:+ demote promotable resources and stop all other resources in the
++  affected cluster partition
+ * +suicide:+ fence all nodes in the affected cluster partition
+ 
+ | batch-limit | 0 |
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-11-0b68344-Refactor-scheduler-functionize-checking-quorum.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-11-0b68344-Refactor-scheduler-functionize-checking-quorum.patch
 new file mode 100644
index 0000000..ac2e23d
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-11-0b68344-Refactor-scheduler-functionize-checking-quorum.patch
@@ -0,0 +1,61 @@
+From: Ken Gaillot <[e-mail]>
+Date: Tue, 2 Jun 2020 15:05:56 -0500
+Subject: Refactor: scheduler: functionize checking quorum policy in effect
+
+... for readability and ease of future changes
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/0b68344
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ lib/pengine/utils.c | 18 ++++++++++++++----
+ 1 file changed, 14 insertions(+), 4 deletions(-)
+
+diff --git a/lib/pengine/utils.c b/lib/pengine/utils.c
+index e57d858..b842481 100644
+--- a/lib/pengine/utils.c
++++ b/lib/pengine/utils.c
+@@ -451,6 +451,17 @@ sort_rsc_priority(gconstpointer a, gconstpointer b)
+     return 0;
+ }
+ 
++static enum pe_quorum_policy
++effective_quorum_policy(pe_resource_t *rsc, pe_working_set_t *data_set)
++{
++    enum pe_quorum_policy policy = data_set->no_quorum_policy;
++
++    if (is_set(data_set->flags, pe_flag_have_quorum)) {
++        policy = no_quorum_ignore;
++    }
++    return policy;
++}
++
+ action_t *
+ custom_action(resource_t * rsc, char *key, const char *task,
+               node_t * on_node, gboolean optional, gboolean save_action,
+@@ -554,6 +565,7 @@ custom_action(resource_t * rsc, char *key, const char *task,
+ 
+     if (rsc != NULL) {
+         enum action_tasks a_task = text2task(action->task);
++        enum pe_quorum_policy quorum_policy = effective_quorum_policy(rsc, data_set);
+         int warn_level = LOG_TRACE;
+ 
+         if (save_action) {
+@@ -625,13 +637,11 @@ custom_action(resource_t * rsc, char *key, const char *task,
+             crm_trace("Action %s requires only stonith", action->uuid);
+             action->runnable = TRUE;
+ #endif
+-        } else if (is_set(data_set->flags, pe_flag_have_quorum) == FALSE
+-                   && data_set->no_quorum_policy == no_quorum_stop) {
++        } else if (quorum_policy == no_quorum_stop) {
+             pe_action_set_flag_reason(__FUNCTION__, __LINE__, action, NULL, "no quorum", pe_action_runnable, TRUE);
+             crm_debug("%s\t%s (cancelled : quorum)", action->node->details->uname, action->uuid);
+ 
+-        } else if (is_set(data_set->flags, pe_flag_have_quorum) == FALSE
+-                   && data_set->no_quorum_policy == no_quorum_freeze) {
++        } else if (quorum_policy == no_quorum_freeze) {
+             pe_rsc_trace(rsc, "Check resource is already active: %s %s %s %s", rsc->id, action->uuid, role2text(rsc->next_role), role2text(rsc->role));
+             if (rsc->fns->active(rsc, TRUE) == FALSE || rsc->next_role > rsc->role) {
+                 pe_action_set_flag_reason(__FUNCTION__, __LINE__, action, NULL, "quorum freeze", pe_action_runnable, TRUE);
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-12-b1ae359-Feature-scheduler-support-demote-choice-for.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-12-b1ae359-Feature-scheduler-support-demote-choice-for.patch
 new file mode 100644
index 0000000..c7fc280
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-12-b1ae359-Feature-scheduler-support-demote-choice-for.patch
@@ -0,0 +1,163 @@
+From: Ken Gaillot <[e-mail]>
+Date: Tue, 2 Jun 2020 15:06:32 -0500
+Subject: Feature: scheduler: support "demote" choice for no-quorum-policy
+ option
+
+If quorum is lost, promotable resources in the master role will be demoted but
+left running, and all other resources will be stopped.
+
+  [Backport]
+
+  Existing pacemaker version did not have create common/options.c yet,
+  and did not have a detailed pengine/pe_output.c txt output function.
+  I have changed html and xml output functions (as the original patch)
+  and kept the same changes in everything else.
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+
+Author: Ken Gaillot <[e-mail]>
+Origin: backport, https://github.com/ClusterLabs/pacemaker/commit/b1ae359
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ daemons/controld/controld_control.c |  2 +-
+ include/crm/pengine/pe_types.h      |  3 ++-
+ lib/common/utils.c                  |  3 +++
+ lib/pengine/common.c                |  2 +-
+ lib/pengine/unpack.c                |  7 +++++++
+ lib/pengine/utils.c                 | 14 ++++++++++++++
+ tools/crm_mon_output.c              |  9 +++++++++
+ 7 files changed, 37 insertions(+), 3 deletions(-)
+
+diff --git a/daemons/controld/controld_control.c b/daemons/controld/controld_control.c
+index 6c7f97c..132b059 100644
+--- a/daemons/controld/controld_control.c
++++ b/daemons/controld/controld_control.c
+@@ -587,7 +587,7 @@ static pe_cluster_option crmd_opts[] = {
+         { "stonith-max-attempts",NULL,"integer",NULL,"10",&check_positive_number,
+           "How many times stonith can fail before it will no longer be attempted on a target"
+         },   
+-	{ "no-quorum-policy", NULL, "enum", "stop, freeze, ignore, suicide", "stop", &check_quorum, NULL, NULL },
++	{ "no-quorum-policy", NULL, "enum", "stop, freeze, ignore, demote, suicide", "stop", &check_quorum, NULL, NULL },
+ };
+ /* *INDENT-ON* */
+ 
+diff --git a/include/crm/pengine/pe_types.h b/include/crm/pengine/pe_types.h
+index 6e5cbcc..baa9160 100644
+--- a/include/crm/pengine/pe_types.h
++++ b/include/crm/pengine/pe_types.h
+@@ -61,7 +61,8 @@ enum pe_quorum_policy {
+     no_quorum_freeze,
+     no_quorum_stop,
+     no_quorum_ignore,
+-    no_quorum_suicide
++    no_quorum_suicide,
++    no_quorum_demote
+ };
+ 
+ enum node_type {
+diff --git a/lib/common/utils.c b/lib/common/utils.c
+index cb0bc1f..b114b44 100644
+--- a/lib/common/utils.c
++++ b/lib/common/utils.c
+@@ -140,6 +140,9 @@ check_quorum(const char *value)
+     } else if (safe_str_eq(value, "ignore")) {
+         return TRUE;
+ 
++    } else if (safe_str_eq(value, "demote")) {
++        return TRUE;
++
+     } else if (safe_str_eq(value, "suicide")) {
+         return TRUE;
+     }
+diff --git a/lib/pengine/common.c b/lib/pengine/common.c
+index fcd7cf0..d134d79 100644
+--- a/lib/pengine/common.c
++++ b/lib/pengine/common.c
+@@ -75,7 +75,7 @@ check_placement_strategy(const char *value)
+ /* *INDENT-OFF* */
+ static pe_cluster_option pe_opts[] = {
+ 	/* name, old-name, validate, default, description */
+-	{ "no-quorum-policy", NULL, "enum", "stop, freeze, ignore, suicide", "stop", &check_quorum,
++	{ "no-quorum-policy", NULL, "enum", "stop, freeze, ignore, demote, suicide", "stop", &check_quorum,
+ 	  "What to do when the cluster does not have quorum", NULL },
+ 	{ "symmetric-cluster", NULL, "boolean", NULL, "true", &check_boolean,
+ 	  "All resources can run anywhere by default", NULL },
+diff --git a/lib/pengine/unpack.c b/lib/pengine/unpack.c
+index e690c4e..3306662 100644
+--- a/lib/pengine/unpack.c
++++ b/lib/pengine/unpack.c
+@@ -243,6 +243,9 @@ unpack_config(xmlNode * config, pe_working_set_t * data_set)
+     } else if (safe_str_eq(value, "freeze")) {
+         data_set->no_quorum_policy = no_quorum_freeze;
+ 
++    } else if (safe_str_eq(value, "demote")) {
++        data_set->no_quorum_policy = no_quorum_demote;
++
+     } else if (safe_str_eq(value, "suicide")) {
+         if (is_set(data_set->flags, pe_flag_stonith_enabled)) {
+             int do_panic = 0;
+@@ -271,6 +274,10 @@ unpack_config(xmlNode * config, pe_working_set_t * data_set)
+         case no_quorum_stop:
+             crm_debug("On loss of quorum: Stop ALL resources");
+             break;
++        case no_quorum_demote:
++            crm_debug("On loss of quorum: "
++                      "Demote promotable resources and stop other resources");
++            break;
+         case no_quorum_suicide:
+             crm_notice("On loss of quorum: Fence all remaining nodes");
+             break;
+diff --git a/lib/pengine/utils.c b/lib/pengine/utils.c
+index b842481..fd06e9e 100644
+--- a/lib/pengine/utils.c
++++ b/lib/pengine/utils.c
+@@ -458,6 +458,20 @@ effective_quorum_policy(pe_resource_t *rsc, pe_working_set_t *data_set)
+ 
+     if (is_set(data_set->flags, pe_flag_have_quorum)) {
+         policy = no_quorum_ignore;
++
++    } else if (data_set->no_quorum_policy == no_quorum_demote) {
++        switch (rsc->role) {
++            case RSC_ROLE_MASTER:
++            case RSC_ROLE_SLAVE:
++                if (rsc->next_role > RSC_ROLE_SLAVE) {
++                    rsc->next_role = RSC_ROLE_SLAVE;
++                }
++                policy = no_quorum_ignore;
++                break;
++            default:
++                policy = no_quorum_stop;
++                break;
++        }
+     }
+     return policy;
+ }
+diff --git a/tools/crm_mon_output.c b/tools/crm_mon_output.c
+index c27aa83..bb6b8b8 100644
+--- a/tools/crm_mon_output.c
++++ b/tools/crm_mon_output.c
+@@ -472,6 +472,11 @@ cluster_options_html(pcmk__output_t *out, va_list args) {
+             out->list_item(out, NULL, "No Quorum policy: Stop ALL resources");
+             break;
+ 
++        case no_quorum_demote:
++            out->list_item(out, NULL, "No Quorum policy: Demote promotable "
++                           "resources and stop all other resources");
++            break;
++
+         case no_quorum_ignore:
+             out->list_item(out, NULL, "No Quorum policy: Ignore");
+             break;
+@@ -526,6 +531,10 @@ cluster_options_xml(pcmk__output_t *out, va_list args) {
+             xmlSetProp(node, (pcmkXmlStr) "no-quorum-policy", (pcmkXmlStr) "stop");
+             break;
+ 
++        case no_quorum_demote:
++            xmlSetProp(node, (pcmkXmlStr) "no-quorum-policy", (pcmkXmlStr) "demote");
++            break;
++
+         case no_quorum_ignore:
+             xmlSetProp(node, (pcmkXmlStr) "no-quorum-policy", (pcmkXmlStr) "ignore");
+             break;
diff --git a/debian/patches/ubuntu-2.0.3-demote/lp1896223-13-d4b9117-Doc-Pacemaker-Explained-correct-on-fail-default.patch b/debian/patches/ubuntu-2.0.3-demote/lp1896223-13-d4b9117-Doc-Pacemaker-Explained-correct-on-fail-default.patch
 new file mode 100644
index 0000000..f70ac52
--- /dev/null
+++ b/debian/patches/ubuntu-2.0.3-demote/lp1896223-13-d4b9117-Doc-Pacemaker-Explained-correct-on-fail-default.patch
@@ -0,0 +1,33 @@
+From: Ken Gaillot <[e-mail]>
+Date: Tue, 26 May 2020 18:10:33 -0500
+Subject: Doc: Pacemaker Explained: correct on-fail default
+
+Author: Ken Gaillot <[e-mail]>
+Origin: upstream, https://github.com/ClusterLabs/pacemaker/commit/d4b9117
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/1896223
+Reviewed-by: Rafael David Tinoco <[e-mail]>
+Last-Update: 2020-10-05
+---
+ doc/Pacemaker_Explained/en-US/Ch-Resources.txt | 9 +++++++--
+ 1 file changed, 7 insertions(+), 2 deletions(-)
+
+diff --git a/doc/Pacemaker_Explained/en-US/Ch-Resources.txt b/doc/Pacemaker_Explained/en-US/Ch-Resources.txt
+index 9df9243..88892db 100644
+--- a/doc/Pacemaker_Explained/en-US/Ch-Resources.txt
++++ b/doc/Pacemaker_Explained/en-US/Ch-Resources.txt
+@@ -669,8 +669,13 @@ XML attributes take precedence over +nvpair+ elements if both are specified.
+  indexterm:[Action,Property,timeout]
+ 
+ |on-fail
+-|restart '(except for +stop+ operations, which default to' fence 'when
+- STONITH is enabled and' block 'otherwise)'
++a|Varies by action:
++
++* +stop+: +fence+ if +stonith-enabled+ is true or +block+ otherwise
++* +demote+: +on-fail+ of the +monitor+ action with +role+ set to +Master+, if
++  present, enabled, and configured to a value other than +demote+, or +restart+
++  otherwise
++* all other actions: +restart+
+ a|The action to take if this action ever fails. Allowed values:
+ 
+ * +ignore:+ Pretend the resource did not fail.