Merge ~rafaeldtinoco/ubuntu/+source/samba:lp1831381-eoan-devel-ctdb into ubuntu/+source/samba:ubuntu/eoan-devel

[Rafael David Tinoco]

BUG taking care of all the CTDB bugs related to enablement and NFS enablement:

https://bugs.launchpad.net/ubuntu-server-ha/+bug/1831381

Other bugs (samba/apache/... resources) will be taken care in other bugs inside:

https://bugs.launchpad.net/ubuntu-server-ha/

in the appropriate time.

---- Obs ----

I have fully tested the PPA for Eoan and the documentation is here:

https://blueprints.launchpad.net/ubuntu-server-ha/+spec/ctdb-enablement-nfs

And the PPA:

https://launchpad.net/~rafaeldtinoco/+archive/ubuntu/lp1831381

And the git repo:

https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/samba/+git/samba/+ref/lp1831381-eoan-devel-ctdb

I'm changing the public bugs for the SRU template now AND will submit-to-debian as soon as this review is over (before attempting the SRUs).

[Andreas Hasenack (ahasenack)]

Thanks for all this work!

Taking a first pass.

needs-info about +ctdb-scripts-fix-60-nfs-service-name.patch

Will continue the review.

review: Needs Information

[Andreas Hasenack (ahasenack)]

Would it make sense to merge ctdb-scripts-change-callout-files-for-debian-and-ubuntu.patch and ctdb-scripts-fix-60-nfs-service-name.patch? Both are about changing the service names for Ubuntu, and point at the same bug both in ubuntu and upstream (#13860)

[Rafael David Tinoco (rafaeldtinoco)]

Fixing after review, will push it again. Tks a lot for reviewing/sponsoring it for me!

[Andreas Hasenack (ahasenack)]

Continuing review.

[Andreas Hasenack (ahasenack)]

Comment inline, will continue tomorrow. Thanks so far!

[Andreas Hasenack (ahasenack)]

Few more comments

[Rafael David Tinoco (rafaeldtinoco)]

Dropping:

commit 782ee0afdec549f63bfe3e6abdfe2bdce6039316 (HEAD)
Author: Rafael David Tinoco <email address hidden>
Date: Fri Jun 7 00:47:56 2019

      * d/control: added nfs and quota as recommended packages since CTDB
        NFS HA scripts, when enabled, need them (LP: #722201)

    Signed-off-by: Rafael David Tinoco <email address hidden>

diff --git a/debian/control b/debian/control
index d9fd63cac..c915ed110 100644
--- a/debian/control
+++ b/debian/control
@@ -437,7 +437,10 @@ Depends: iproute2 [linux-any],
          time,
          ${misc:Depends},
          ${shlibs:Depends}
-Recommends: ethtool [linux-any]
+Recommends: ethtool [linux-any],
+ nfs-kernel-server,
+ nfs-common,
+ quota
 Suggests: logrotate, lsof
 Description: clustered database to store temporary data
  CTDB is a cluster implementation of the TDB database used by Samba and other

Based on in-person discussion we had.

[Rafael David Tinoco (rafaeldtinoco)]

Some inline responses while doing the rebase

[Rafael David Tinoco (rafaeldtinoco)]

$ sudo dpkg -i ./ctdb_4.10.0+dfsg-0ubuntu4_amd64.deb
Selecting previously unselected package ctdb.
(Reading database ... 129258 files and directories currently installed.)
Preparing to unpack .../ctdb_4.10.0+dfsg-0ubuntu4_amd64.deb ...
Unpacking ctdb (2:4.10.0+dfsg-0ubuntu4) ...
Setting up ctdb (2:4.10.0+dfsg-0ubuntu4) ...
Created symlink /etc/systemd/system/multi-user.target.wants/ctdb.service → /lib/systemd/system/ctdb.service.
Processing triggers for libc-bin (2.29-0ubuntu2) ...
Processing triggers for systemd (240-6ubuntu9) ...
Processing triggers for man-db (2.8.5-2) ...

(c)inaddy@ctdbdevel:~/work/sources/ubuntu/git$ systemctl enable ctdb
Synchronizing state of ctdb.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable ctdb
(c)inaddy@ctdbdevel:~/work/sources/ubuntu/git$ systemctl start ctdb
(c)inaddy@ctdbdevel:~/work/sources/ubuntu/git$ systemctl status ctdb
● ctdb.service - CTDB
   Loaded: loaded (/lib/systemd/system/ctdb.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-06-27 12:19:20 UTC; 1s ago
     Docs: man:ctdbd(1)
           man:ctdb(7)
  Process: 26458 ExecStart=/usr/sbin/ctdbd_wrapper start (code=exited, status=0/SUCCESS)
 Main PID: 26460 (ctdbd)
    Tasks: 3 (limit: 4915)
   Memory: 6.1M
   CGroup: /system.slice/ctdb.service
           ├─26460 /usr/sbin/ctdbd
           ├─26462 /usr/lib/x86_64-linux-gnu/ctdb/ctdb-eventd -P 26460 -S 14
           └─26492 /usr/sbin/ctdbd

Jun 27 12:19:19 ctdbdevel systemd[1]: Starting CTDB...
Jun 27 12:19:19 ctdbdevel ctdbd[26459]: CTDB logging to location file:/var/log/ctdb/log.ctdb
Jun 27 12:19:20 ctdbdevel systemd[1]: Started CTDB.

(c)inaddy@ctdbdevel:~/work/sources/ubuntu/git$ dpkg -L ctdb > ctdb.files

(c)inaddy@ctdbdevel:~/work/sources/ubuntu/git$ apt-get remove --purge ctdb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following packages were automatically installed and are no longer required:
  ethtool libwbclient0 samba-libs tdb-tools time
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  ctdb*
0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
After this operation, 7,854 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 129371 files and directories currently installed.)
Removing ctdb (2:4.10.0+dfsg-0ubuntu4) ...
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for libc-bin (2.29-0ubuntu2) ...
(Reading database ... 129288 files and directories currently installed.)
Purging configuration files for ctdb (2:4.10.0+dfsg-0ubuntu4) ...
dpkg: warning: while removing ctdb, directory '/var/log/ctdb' not empty so not removed
dpkg: warning: while removing ctdb, directory '/var/lib/ctdb/state' not empty so not removed
dpkg: warning: while removing ctdb, directory '/var/lib/ctdb/persistent' not empty so not removed
Processing triggers for systemd (240-6ubuntu9) ...

(c)inaddy@ctdbdevel:~/work/sources/ubuntu/git$ for entry in $(cat ctdb.files); do [ -...

[Rafael David Tinoco (rafaeldtinoco)]

Transferred examples to documentation:

$ cat ctdb.files | grep doc
/usr/share/doc
/usr/share/doc/ctdb
/usr/share/doc/ctdb/NEWS.Debian.gz
/usr/share/doc/ctdb/README
/usr/share/doc/ctdb/README.eventscripts.gz
/usr/share/doc/ctdb/README.notification
/usr/share/doc/ctdb/changelog.Debian.gz
/usr/share/doc/ctdb/cluster_mutex_helper.txt
/usr/share/doc/ctdb/copyright
/usr/share/doc/ctdb/examples
/usr/share/doc/ctdb/examples/11.natgw.options
/usr/share/doc/ctdb/examples/20.nfs_ganesha.check
/usr/share/doc/ctdb/examples/91.lvs.options
/usr/share/doc/ctdb/examples/README
/usr/share/doc/ctdb/examples/config_migrate.sh.gz
/usr/share/doc/ctdb/examples/config_migrate.test_input
/usr/share/doc/ctdb/examples/ctdb.conf
/usr/share/doc/ctdb/examples/nfs-ganesha-callout.gz
/usr/share/doc/ctdb/examples/nfs-kernel-server
/usr/share/doc/ctdb/examples/nfs-kernel-server/99-nfs-static-pors.conf
/usr/share/doc/ctdb/examples/nfs-kernel-server/enable-nfs.sh.gz
/usr/share/doc/ctdb/examples/nfs-kernel-server/nfs-common
/usr/share/doc/ctdb/examples/nfs-kernel-server/nfs-kernel-server
/usr/share/doc/ctdb/examples/nfs-kernel-server/services
/usr/share/doc/ctdb/readonlyrecords.txt.gz
/usr/share/doc/ctdb/recovery-process.txt.gz

Testing scripts now...

[Rafael David Tinoco (rafaeldtinoco)]

Current status:

REVIEW being done:

  * d/p/ctdb-scripts-fix-tcp_tw_recycle-existence-check.patch: fix
    tcp_tw_recycle existence check. (LP: #722201)

  ok

  * d/p/fix-nfs-service-name-to-nfs-kernel-server.patch: change nfs
    service name from nfs to nfs-kernel-server (LP: #722201)

  ok - check quilt patch description (upstream svc name is ok)

  * d/ctdb.install, d/rules: create ctdb run directory into tmpfiles.d
    to allow pid file to exist (LP: #1821775)

  - dpkg: warning: while removing ctdb, directory '/var/lib/ctdb/state' not empty so not removed
  - dpkg: warning: while removing ctdb, directory '/var/lib/ctdb/persistent' not empty so not removed

  * d/ctdb.dirs: added /var/lib/ctdb/* directories to allow proper ctdb
    initialization. (LP: #1828799)

  ok

  * d/ctdb.postinst, d/ctdb.postrm: allow service to start by default in
    a single localhost node setup with no errors. (LP: #722201)

  - do not start service by default

  * NEW PATCH

   1- make systemd service to depend on:
      - /etc/ctdb/nodes

   2- change logging to syslog (journal -f -u ctdb will work)

  * Examples of NFS HA CTDB config files + helper script:
    - d/ctdb.example.enable.nfs.sh
    - d/ctdb.example.nfs-common
    - d/ctdb.example.nfs-kernel-server
    - d/ctdb.example.services
    - d/ctdb.example.sysctl-nfs-static-ports.conf

[Rafael David Tinoco (rafaeldtinoco)]

Alright, I think I got all we have discussed covered. I have tested the .sh script by gunziping it in ~/ and executing it with sudo. I'm providing new instructions in a post in discourse and will point the old blueprint to that post (much better interface until we don't have documentation place to concentrate docs).

[Rafael David Tinoco (rafaeldtinoco)]

Provided new documentation in here:

https://discourse.ubuntu.com/t/ctdb-create-a-3-node-nfs-ha-backed-by-a-clustered-filesystem/11608

[Rafael David Tinoco (rafaeldtinoco)]

PPA failed because of a leftover I had as a workaround to libaesni case:

dh_missing: usr/lib/x86_64-linux-gnu/samba/libaesni-intel.so.0 exists in debian/tmp but is not installed to anywhere

I'm fixing it and re-pushing.

[Andreas Hasenack (ahasenack)]

Hello CTDB, we meet again!

aka, /me continues the review

[Andreas Hasenack (ahasenack)]

Very nice explanation on debian/patches/fix-nfs-service-name-to-nfs-kernel-server.patch, thanks for that.

Other comments inline, about examples and how to install them, and the post script which I think you changed already (we talked on irc), but I didn't reload the page yet.

[Rafael David Tinoco (rafaeldtinoco)]

Okay, sorry for the huge delay in between the pushes. I'll be more careful about that from now on. Let me handle what you've suggested in this review. Starting over again...

[Rafael David Tinoco (rafaeldtinoco)]

I'm fixing things you pointed AND also something I forgot about last push.

The following fix I sent upstream:

https://salsa.debian.org/samba-team/samba/merge_requests/24#note_93728

And I'm pushing it again for the Salsa CI to work.

[Rafael David Tinoco (rafaeldtinoco)]

Thanks again for reviewing this Andreas. Be sure I'll provide quick feedback if needed and, tomorrow, I'll be available, if you want to have a hangout (or in-office meeting), so we can finish this.

[Rafael David Tinoco (rafaeldtinoco)]

I uploaded this new version to the PPA as well!

[Andreas Hasenack (ahasenack)]

Checking it out, thanks for the update

[Rafael David Tinoco (rafaeldtinoco)]

Replied to your suggestions, depending on the answer I'll work one way or another. Tks!

[Rafael David Tinoco (rafaeldtinoco)]

Answers, new push to git and dput to ppa.

[Rafael David Tinoco (rafaeldtinoco)]

14:42 <andreas> rafaeldtinoco: bug upstream with patch attached, and we continue on our own while they check that
14:42 <andreas> no need to block
14:43 <andreas> and record the upstream bug in the dep3 header
14:43 <rafaeldtinoco> yep just added a comment on DEP3
14:43 <rafaeldtinoco> (forward to upstream opened)
14:43 <rafaeldtinoco> and will submit to debian together

14:58 <rafaeldtinoco> andreas: re-pushed
14:58 <rafaeldtinoco> https://bugzilla.samba.org/show_bug.cgi?id=14026
14:58 <rafaeldtinoco> opened upstream, attached patch
14:58 <rafaeldtinoco> mailed patch to samba mailing list
14:59 <rafaeldtinoco> added DEP3 forwarded to our patch before push

[Rafael David Tinoco (rafaeldtinoco)]

I fixed and I'm pushing again the first comment. Asked you a few things on the second.

[Andreas Hasenack (ahasenack)]

Thanks for the responses.

I'm EOD now, and will finish this tomorrow. Cheers!

[Andreas Hasenack (ahasenack)]

Nice, +1!

review: Approve

[Andreas Hasenack (ahasenack)]

Sponsoring 44d89a65cc17b45a65ec131f53243a47d9933001

Tagged and uploaded:

$ git push pkg upload/2%4.10.0+dfsg-0ubuntu5
Enumerating objects: 61, done.
Counting objects: 100% (61/61), done.
Delta compression using up to 2 threads
Compressing objects: 100% (52/52), done.
Writing objects: 100% (52/52), 11.68 KiB | 221.00 KiB/s, done.
Total 52 (delta 32), reused 0 (delta 0)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/samba
 * [new tag] upload/2%4.10.0+dfsg-0ubuntu5 -> upload/2%4.10.0+dfsg-0ubuntu5

$ dput ubuntu ../samba_4.10.0+dfsg-0ubuntu5_source.changes
Checking signature on .changes
gpg: ../samba_4.10.0+dfsg-0ubuntu5_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../samba_4.10.0+dfsg-0ubuntu5.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading samba_4.10.0+dfsg-0ubuntu5.dsc: done.
  Uploading samba_4.10.0+dfsg-0ubuntu5.debian.tar.xz: done.
  Uploading samba_4.10.0+dfsg-0ubuntu5_source.buildinfo: done.
  Uploading samba_4.10.0+dfsg-0ubuntu5_source.changes: done.
Successfully uploaded packages.

diff --git a/debian/changelog b/debian/changelog
index a64b76a..0e180bf 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,32 @@
+samba (2:4.10.0+dfsg-0ubuntu5) eoan; urgency=medium
+
+  * debian/rules: Make DEB_HOST_ARCH_CPU initialized through
+    dpkg-architecture (Closes: #931138)
+  * d/p/ctdb-scripts-fix-tcp_tw_recycle-existence-check.patch:
+    fix tcp_tw_recycle existence check. (LP: #722201)
+  * d/p/fix-nfs-service-name-to-nfs-kernel-server.patch:
+    change nfs service name from nfs to nfs-kernel-server
+    (LP: #722201)
+  * d/ctdb.install, d/rules: create ctdb run directory into tmpfiles.d
+    to allow pid file to exist (LP: #1821775)
+  * Allow proper ctdb initialization (LP: #1828799):
+    - d/ctdb.dirs: added /var/lib/ctdb/* directories
+    - d/ctdb.postrm: remove leftovers from:
+      /var/lib/ctdb/{state,persistent,volatile,scripts}
+  * d/rules: installing provided config examples and helper scripts
+  * Examples of NFS HA CTDB config files + helper script:
+    - d/ctdb.example.enable.nfs.sh
+    - d/ctdb.example.nfs-common
+    - d/ctdb.example.nfs-kernel-server
+    - d/ctdb.example.services
+    - d/ctdb.example.sysctl-nfs-static-ports.conf
+  * d/p/ctdb-config-depend-on-etc-default-nodes-file.patch:
+    do not try to start daemon if /etc/ctdb/nodes does not exist
+  * d/p/ctdb-config-enable-syslog-by-default.patch:
+    enable syslog and systemd journal by default
+
+ -- Rafael David Tinoco <[e-mail]>  Fri, 28 Jun 2019 00:14:27 +0000
+
 samba (2:4.10.0+dfsg-0ubuntu4) eoan; urgency=medium
 
   * SECURITY UPDATE: zone operations can crash rpc server
diff --git a/debian/ctdb.dirs b/debian/ctdb.dirs
index 2c65b74..3907a72 100644
--- a/debian/ctdb.dirs
+++ b/debian/ctdb.dirs
@@ -1 +1,5 @@
 var/log/ctdb
+var/lib/ctdb
+var/lib/ctdb/persistent
+var/lib/ctdb/state
+var/lib/ctdb/volatile
diff --git a/debian/ctdb.example.enable.nfs.sh b/debian/ctdb.example.enable.nfs.sh
new file mode 100755
index 0000000..2a99074
--- /dev/null
+++ b/debian/ctdb.example.enable.nfs.sh
@@ -0,0 +1,196 @@
+#!/bin/bash -e
+
+base="/usr/share/doc/ctdb/examples/nfs-kernel-server/"
+logfile="/tmp/enable-ctdb-nfs.$$.log" ; touch $logfile ;
+ghostname=""
+
+# functions ---------
+
+die() { echo error: $@; echo ; exit 1; };
+getout() { echo exit: $@; echo ; exit 0; };
+stopservice() { echo stopping $1... ; systemctl stop $1 2>&1 >> $logfile 2>&1; }
+disableservice() { echo disabling $1... ; systemctl disable $1 2>&1 >> $logfile 2>&1; }
+startservice() { echo starting $1... ; systemctl start $1 2>&1 >> $logfile 2>&1; }
+sysctlrefresh() { echo refreshing sysctl... ; sysctl --system 2>&1 >> $logfile 2>&1; }
+
+backupfile() {
+    echo backing up $1
+    [ -f $1.prvctdb ] && die "backup file $1 already exists!"
+    [ -f $1 ] && cp $1 $1.prvctdb || true
+}
+
+checkservice() {
+    (systemctl list-unit-files | grep -q $1.service) || die "service $1 not found"
+}
+
+replacefile() {
+
+    origfile=$1
+    replfile=$2
+
+
+    [ ! -f $base/$origfile ] && die "coult not find $base/$origfile"
+
+    echo replacing $replfile...
+    cp $base/$origfile $replfile
+}
+
+appendfile() {
+
+    origfile=$1
+    replfile=$2
+
+    [ ! -f $base/$origfile ] && die "coult not find $base/$origfile"
+
+    echo appending $base/$origfile to $replfile...
+    cat $base/$origfile >> $replfile
+}
+
+appendnfsenv() {
+
+    file=$1 ; [ -f $file ] || die "inexistent file $file";
+
+    echo appending NFS_HOSTNAME to $file...
+
+    grep -q "NFS_HOSTNAME" $file || \
+    {
+        echo
+        echo "echo NFS_HOSTNAME=\\\"\$NFS_HOSTNAME\\\"" \>\> \/run\/sysconfig\/nfs-utils
+        echo
+    } >> $file
+}
+
+execnfsenv() {
+
+    file=$1 ; [ -f $file ] || due "inexistent file $file";
+
+    echo executing $file...
+
+    $file 2>&1 >> $logfile 2>&1;
+}
+
+fixnfshostname() {
+
+    file=$1 ; [ -f $file ] || due "inexistent file $file";
+
+    if [ "$ghostname" == "" ]; then
+        echo "What is the FQDN for the public IP address of this host ?"
+        echo -n "> "
+        read ghostname
+    fi
+
+    echo placing hostname $ghostname into $file...
+    sed -i "s:PLACE_HOSTNAME_HERE:$ghostname:g" $file
+}
+
+# end of functions --
+
+[ $UID != 0 ] && die "you need root privileges"
+
+echo """
+This script will enable CTDB NFS HA by changing the following files:
+
+(1) /etc/default/nfs-common                   ( replace )
+(2) /etc/default/nfs-kernel-server            ( replace )
+(3) /etc/services                             ( append  )
+(4) /etc/sysctl.d/99-nfs-static-ports.conf    ( create  )
+(5) /usr/lib/systemd/scripts/nfs-utils_env.sh ( modify  )
+
+and disabling the following services:
+
+(1) rpcbind
+(2) nfs-kernel-server
+(3) rpc.rquotad
+
+Obs:
+  - replaced files keep previous versions as "file".prevctdb
+  - dependant services will also be stopped
+"""
+
+while true; do
+    echo -n "Do you agree with this change ? (N/y) => "
+    read answer
+    [ "$answer" == "n" ] && getout "exiting without any changes"
+    [ "$answer" == "y" ] && break
+done
+
+
+echo "checking requirements..."
+
+checkservice nfs-kernel-server
+checkservice quota
+checkservice rpcbind
+
+echo "requirements okay!"
+echo
+
+backupfile /etc/default/nfs-common
+backupfile /etc/default/nfs-kernel-server
+backupfile /etc/services
+backupfile /usr/lib/systemd/scripts/nfs-utils_env.sh
+echo
+
+set +e
+
+stopservice ctdb.service
+stopservice quota.service
+stopservice nfs-kernel-server.service
+stopservice rpcbind.service
+stopservice rpcbind.socket
+stopservice rpcbind.target
+echo
+
+disableservice ctdb.service
+disableservice quota.service
+disableservice nfs-kernel-server.service
+disableservice rpcbind.service
+disableservice rpcbind.socket
+disableservice rpcbind.target
+echo
+
+set -e
+
+replacefile nfs-common /etc/default/nfs-common
+replacefile nfs-kernel-server /etc/default/nfs-kernel-server
+replacefile 99-nfs-static-ports.conf /etc/sysctl.d/99-nfs-static-ports.conf
+echo
+
+appendfile services /etc/services
+echo
+
+fixnfshostname /etc/default/nfs-common
+fixnfshostname /etc/default/nfs-kernel-server
+echo
+
+appendnfsenv /usr/lib/systemd/scripts/nfs-utils_env.sh
+execnfsenv /usr/lib/systemd/scripts/nfs-utils_env.sh
+echo
+
+sysctlrefresh
+echo
+
+echo """Finished! Make sure to configure properly:
+
+    - /etc/exports (containing the clustered fs to be exported)
+    - /etc/ctdb/nodes (containing all your node private IPs)
+    - /etc/ctdb/public_addressess (containing public addresses)
+
+A log file can be found at:
+
+    - /tmp/enable-ctdb-nfs.$$.log
+
+Remember:
+
+    - to place a recovery lock in /etc/ctdb/ctdb.conf:
+        ...
+        [cluster]
+        recovery lock = /clustered.filesystem/.reclock
+        ...
+
+And, make sure you enable ctdb service again:
+
+    - systemctl enable ctdb.service
+    - systemctl start ctdb.service
+
+Enjoy!
+"""
diff --git a/debian/ctdb.example.nfs-common b/debian/ctdb.example.nfs-common
new file mode 100644
index 0000000..9d4f22c
--- /dev/null
+++ b/debian/ctdb.example.nfs-common
@@ -0,0 +1,19 @@
+# CTDB: /etc/default/nfs-common for clustering
+
+NFS_HOSTNAME="PLACE_HOSTNAME_HERE"
+
+# rpc.statd - daemon listening for reboot notifications (locks related)
+NEED_STATD="yes"
+STATDOPTS="-n ${NFS_HOSTNAME} -p 32765 -o 32766 -H /etc/ctdb/statd-callout -T 32768 -U 32768"
+STATD_HOSTNAME="$NFS_HOSTNAME"
+
+# rpc.gssd - security context for rpc connections
+NEED_GSSD="no"
+
+# rpc.idmapd - NFSv4 <-> name mapping daemon (fallback nowadays)
+# recent kernels use nfsidmap(8) instead
+NEED_IDMAPD="no"
+
+# rpc.quota - usage quota
+RPCRQUOTADOPTS="-p 32769"
+
diff --git a/debian/ctdb.example.nfs-kernel-server b/debian/ctdb.example.nfs-kernel-server
new file mode 100644
index 0000000..6aa5df9
--- /dev/null
+++ b/debian/ctdb.example.nfs-kernel-server
@@ -0,0 +1,16 @@
+# CTDB: /etc/default/nfs-kernel-server for clustering
+
+NFS_HOSTNAME="PLACE_HOSTNAME_HERE"
+
+# rpc.nfsd - user level part of nfs service (kernel: nfsd module)
+RPCNFSDPRIORITY=0
+RPCNFSDCOUNT=8
+RPCNFSDOPTS="-N 4"
+
+# rpc.mountd - server side of nfs mount protocol
+RPCMOUNTDOPTS="-p 32767 --manage-gids --no-nfs-version 4"
+
+# rpc.svcgssd - userspace daemon to handle sec context for kernel rpcsec_gss
+NEED_SVCGSSD="no"
+RPCSVCGSSDOPTS=""
+
diff --git a/debian/ctdb.example.services b/debian/ctdb.example.services
new file mode 100644
index 0000000..3ff6b1d
--- /dev/null
+++ b/debian/ctdb.example.services
@@ -0,0 +1,16 @@
+# CTDB: fixed NFS/RPC service ports for clustering
+
+rpc.nfsd	2049/tcp	# RPC nfsd
+rpc.nfsd	2049/udp	# RPC nfsd
+rpc.nfs-cb	32764/tcp	# RPC nfs callback
+rpc.nfs-cb	32764/udp	# RPC nfs callback
+rpc.statd-bc	32765/tcp	# RPC statd broadcast
+rpc.statd-bc	32765/udp	# RPC statd broadcast
+rpc.statd	32766/tcp	# RPC statd listen
+rpc.statd	32766/udp	# RPC statd listen
+rpc.mountd	32767/tcp	# RPC mountd
+rpc.mountd	32767/udp	# RPC mountd
+rpc.lockd	32768/tcp	# RPC lockd/nlockmgr
+rpc.lockd	32768/udp	# RPC lockd/nlockmgr
+rpc.quotad	32769/tcp	# RPC quotad
+rpc.quotad	32769/udp	# RPC quotad
diff --git a/debian/ctdb.example.sysctl-nfs-static-ports.conf b/debian/ctdb.example.sysctl-nfs-static-ports.conf
new file mode 100644
index 0000000..7b83623
--- /dev/null
+++ b/debian/ctdb.example.sysctl-nfs-static-ports.conf
@@ -0,0 +1,5 @@
+# CTDB: /etc/sysctl.d/98-nfs-static-ports.conf
+
+fs.nfs.nfs_callback_tcpport = 32764
+fs.nfs.nlm_tcpport = 32768
+fs.nfs.nlm_udpport = 32768
\ No newline at end of file
diff --git a/debian/ctdb.install b/debian/ctdb.install
index efbdde1..4e4cd13 100755
--- a/debian/ctdb.install
+++ b/debian/ctdb.install
@@ -24,6 +24,7 @@ usr/lib/*/ctdb/ctdb_recovery_helper
 usr/lib/*/ctdb/ctdb_takeover_helper
 usr/lib/*/ctdb/smnotify
 usr/lib/*/samba/libctdb-event-client.so.0
+usr/lib/tmpfiles.d/ctdb.conf
 usr/sbin/ctdbd
 usr/sbin/ctdbd_wrapper
 usr/share/ctdb/events/legacy/*.script
diff --git a/debian/ctdb.postrm b/debian/ctdb.postrm
index 221c2bb..3ba81db 100644
--- a/debian/ctdb.postrm
+++ b/debian/ctdb.postrm
@@ -29,9 +29,22 @@ disable_legacy() { # From ctdb/packaging/RPM/ctdb.spec.in
     fi
 }
 
+remove_leftover() {
+
+    # Remove files created after initalization
+
+    rm -f /etc/ctdb/nodes
+    rm -f /etc/ctdb/public_addresses
+    rm -f /var/lib/ctdb/volatile/*
+    rm -f /var/lib/ctdb/state/*
+    rm -f /var/lib/ctdb/persistent/*
+    rm -rf /var/lib/ctdb/scripts
+}
+
 case "$1" in
     purge)
         disable_legacy 0
+        remove_leftover
     ;;
 
     remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear)
diff --git a/debian/patches/ctdb-config-depend-on-etc-default-nodes-file.patch b/debian/patches/ctdb-config-depend-on-etc-default-nodes-file.patch
new file mode 100644
index 0000000..5a7018b
--- /dev/null
+++ b/debian/patches/ctdb-config-depend-on-etc-default-nodes-file.patch
@@ -0,0 +1,36 @@
+ctdb-config: depend on /etc/default/nodes file
+
+CTDB should start as a disabled unit (systemd) in most of the distributions and,
+when trying to enable it for the first time, user should get an unconfigured, or
+similar, error.
+
+Depending on /etc/ctdb/nodes file will give a clear direction to final user on
+what is needed in order to get cluster up and running. It should work like
+previous ENABLED=NO variables in SySV like initialization scripts.
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+-
+Author: Rafael David Tinoco <[e-mail]>
+Bug-Debian: https://bugs.debian.org/929931
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/722201
+Forwarded: https://bugzilla.samba.org/show_bug.cgi?id=14026
+Last-Update: 2018-06-27
+---
+ ctdb/config/ctdb.service | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/ctdb/config/ctdb.service b/ctdb/config/ctdb.service
+index 675b3147417..fd81c38e26d 100644
+--- a/ctdb/config/ctdb.service
++++ b/ctdb/config/ctdb.service
+@@ -2,6 +2,7 @@
+ Description=CTDB
+ Documentation=man:ctdbd(1) man:ctdb(7)
+ After=network-online.target time-sync.target
++ConditionFileNotEmpty=/etc/ctdb/nodes
+ 
+ [Service]
+ Type=forking
+-- 
+2.20.1
+
diff --git a/debian/patches/ctdb-config-enable-syslog-by-default.patch b/debian/patches/ctdb-config-enable-syslog-by-default.patch
new file mode 100644
index 0000000..3e96c5a
--- /dev/null
+++ b/debian/patches/ctdb-config-enable-syslog-by-default.patch
@@ -0,0 +1,49 @@
+ctdb-config: enable syslog by default
+
+CTDB uses /var/log/ctdb/ directory for the default log files. With
+syslog disabled, systemd journal is not able to correctly inform
+errors happening during service initialization.
+
+Upstream community creates generic config files to be used by different
+distributions, so this change makes no big difference to be accepted by
+upstream.
+
+With this patch the end user will be able to identify initialization
+errors by executing:
+
+  systemctl status ctdb.service
+
+or to follow ctdb logs by executing:
+
+  journalctl -f -u ctdb
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+-
+Author: Rafael David Tinoco <[e-mail]>
+Bug-Debian: https://bugs.debian.org/929931
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/722201
+Last-Update: 2018-06-27
+---
+ ctdb/config/ctdb.conf | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/ctdb/config/ctdb.conf b/ctdb/config/ctdb.conf
+index 5440600a4..cde24280e 100644
+--- a/ctdb/config/ctdb.conf
++++ b/ctdb/config/ctdb.conf
+@@ -5,10 +5,10 @@
+ 
+ [logging]
+ 	# Enable logging to syslog
+-	# location = syslog
++	location = syslog
+ 
+ 	# Default log level
+-	# log level = NOTICE
++	log level = NOTICE
+ 
+ [cluster]
+ 	# Shared recovery lock file to avoid split brain.  Daemon
+-- 
+2.20.1
+
diff --git a/debian/patches/ctdb-scripts-fix-tcp_tw_recycle-existence-check.patch b/debian/patches/ctdb-scripts-fix-tcp_tw_recycle-existence-check.patch
new file mode 100644
index 0000000..bef5284
--- /dev/null
+++ b/debian/patches/ctdb-scripts-fix-tcp_tw_recycle-existence-check.patch
@@ -0,0 +1,43 @@
+Description: ctdb-scripts: Fix tcp_tw_recycle existence check
+
+net.ipv4.tcp_tw_recycle has been removed from Linux 4.12 but, still,
+makes sense to check its existence. Unfortunately, current check does
+not test for the procfs file existence. This commit fixes the issue.
+
+BUG: https://bugzilla.samba.org/show_bug.cgi?id=13984
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+Reviewed-by: Martin Schwenke <[e-mail]>
+Reviewed-by: Amitay Isaacs <[e-mail]>
+
+Autobuild-User(master): Andreas Schneider <[e-mail]>
+Autobuild-Date(master): Tue Jun  4 23:31:24 UTC 2019 on sn-devel-184
+
+(cherry picked from commit 843fbb1207ee7ac84f3282974b66b9290d8da0ac)
+
+Author: Rafael David Tinoco <[e-mail]>
+Origin: upstream, https://attachments.samba.org/attachment.cgi?id=15224
+Forwarded: https://lists.samba.org/archive/samba-technical/2019-June/133700.html
+Bug: https://bugzilla.samba.org/show_bug.cgi?id=13984
+Bug-Debian: http://bugs.debian.org/929931
+Bug-Ubuntu: http://bugs.launchpad.net/bugs/722201
+---
+ ctdb/config/nfs-linux-kernel-callout | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/ctdb/config/nfs-linux-kernel-callout b/ctdb/config/nfs-linux-kernel-callout
+index def69a04649..71d8ecf8074 100755
+--- a/ctdb/config/nfs-linux-kernel-callout
++++ b/ctdb/config/nfs-linux-kernel-callout
+@@ -281,8 +281,8 @@ nfs_startup ()
+     basic_stop "nfs" || true
+     basic_start "nfs"
+     _f="${PROCFS_PATH}/sys/net/ipv4/tcp_tw_recycle"
+-    if [ "$_f" ] ; then
+-	echo 1 >"$_f"
++    if [ -f "$_f" ] ; then
++	    echo 1 >"$_f"
+     fi
+ }
+ 
+-- 2.20.1
diff --git a/debian/patches/fix-nfs-service-name-to-nfs-kernel-server.patch b/debian/patches/fix-nfs-service-name-to-nfs-kernel-server.patch
new file mode 100644
index 0000000..c0d8843
--- /dev/null
+++ b/debian/patches/fix-nfs-service-name-to-nfs-kernel-server.patch
@@ -0,0 +1,97 @@
+Description: fix nfs service name to nfs-kernel-server
+
+Upstream code used to comment in/out service script names related to a specific
+distribution. Recently, they've changed this to a method of detecting on which
+distribution ctdb is running, and then setting the systemd unit file name, for
+example.
+
+Because of this new detection mechanism, it is impossible to suggest upstream a
+change like this patch and, at the same time, backporting this new code could
+bring more problems. Instead, this patch should be kept until a new merge with
+upstream is done.
+
+This temporary patch fixes the NFS service name for Debian & Ubuntu.
+
+Signed-off-by: Rafael David Tinoco <[e-mail]>
+
+Author: Rafael David Tinoco <[e-mail]>
+Bug-Debian: https://bugs.debian.org/929931
+Bug-Ubuntu: https://bugs.launchpad.net/bugs/722201
+Last-Update: 2018-06-26
+
+---
+ ctdb/config/events/legacy/60.nfs.script |  4 ++--
+ ctdb/config/nfs-linux-kernel-callout    | 12 ++++++------
+ ctdb/config/statd-callout               |  4 ++--
+ 3 files changed, 10 insertions(+), 10 deletions(-)
+
+diff --git a/ctdb/config/events/legacy/60.nfs.script b/ctdb/config/events/legacy/60.nfs.script
+index 2eb90b421..5c6a09033 100755
+--- a/ctdb/config/events/legacy/60.nfs.script
++++ b/ctdb/config/events/legacy/60.nfs.script
+@@ -6,9 +6,9 @@
+ 
+ . "${CTDB_BASE}/functions"
+ 
+-service_name="nfs"
++service_name="nfs-kernel-server"
+ 
+-load_system_config "nfs"
++load_system_config "nfs-kernel-server"
+ 
+ load_script_options
+ 
+diff --git a/ctdb/config/nfs-linux-kernel-callout b/ctdb/config/nfs-linux-kernel-callout
+index 9b72446b4..3acd43fee 100755
+--- a/ctdb/config/nfs-linux-kernel-callout
++++ b/ctdb/config/nfs-linux-kernel-callout
+@@ -13,9 +13,9 @@ set -e
+ nfs_exports_file="${CTDB_NFS_EXPORTS_FILE:-/var/lib/nfs/etab}"
+ 
+ # Red Hat
+-nfs_service="nfs"
+-nfslock_service="nfslock"
+-nfs_config="/etc/sysconfig/nfs"
++#nfs_service="nfs"
++#nfslock_service="nfslock"
++#nfs_config="/etc/sysconfig/nfs"
+ 
+ # SUSE
+ #nfs_service="nfsserver"
+@@ -23,9 +23,9 @@ nfs_config="/etc/sysconfig/nfs"
+ #nfs_config="/etc/sysconfig/nfs"
+ 
+ # Debian
+-#nfs_service="nfs-kernel-server"
+-#nfslock_service=""
+-#nfs_config="/etc/default/nfs-kernel-server"
++nfs_service="nfs-kernel-server"
++nfslock_service=""
++nfs_config="/etc/default/nfs-kernel-server"
+ 
+ # Override for unit testing
+ if [ -z "$PROCFS_PATH" ] ; then
+diff --git a/ctdb/config/statd-callout b/ctdb/config/statd-callout
+index bf18b5586..d0c807ee3 100755
+--- a/ctdb/config/statd-callout
++++ b/ctdb/config/statd-callout
+@@ -20,14 +20,14 @@ die ()
+     exit 1
+ }
+ 
+-load_system_config "nfs"
++load_system_config "nfs-kernel-server"
+ 
+ [ -n "$NFS_HOSTNAME" ] || \
+     die "NFS_HOSTNAME is not configured. statd-callout failed"
+ 
+ ############################################################
+ 
+-ctdb_setup_state_dir "service" "nfs"
++ctdb_setup_state_dir "service" "nfs-kernel-server"
+ 
+ # script_state_dir set by ctdb_setup_state_dir()
+ # shellcheck disable=SC2154
+-- 
+2.20.1
+
diff --git a/debian/patches/series b/debian/patches/series
index f2c5483..4b5d752 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -18,3 +18,7 @@ CVE-2018-16860-2.patch
 CVE-2019-12435-1.patch
 CVE-2019-12435-2.patch
 CVE-2019-12436.patch
+ctdb-scripts-fix-tcp_tw_recycle-existence-check.patch
+fix-nfs-service-name-to-nfs-kernel-server.patch
+ctdb-config-depend-on-etc-default-nodes-file.patch
+ctdb-config-enable-syslog-by-default.patch
diff --git a/debian/rules b/debian/rules
index 34a5f48..d45700e 100755
--- a/debian/rules
+++ b/debian/rules
@@ -11,6 +11,7 @@ include /usr/share/dpkg/buildflags.mk
 DEB_HOST_ARCH ?= $(shell dpkg-architecture -qDEB_HOST_ARCH)
 DEB_HOST_ARCH_OS ?= $(shell dpkg-architecture -qDEB_HOST_ARCH_OS)
 DEB_HOST_MULTIARCH ?= $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
+DEB_HOST_ARCH_CPU ?= $(shell dpkg-architecture -qDEB_HOST_ARCH_CPU)
 
 LDB_VERSION = $(shell pkg-config --modversion ldb)
 LDB_EPOCH = $(shell dpkg-query -f '$${Version}' -W libldb-dev | sed 's/:.*//')
@@ -169,6 +170,7 @@ endif
 	# such as OpenRC's opentmpfiles
 	mkdir -p $(DESTDIR)/usr/lib/tmpfiles.d
 	echo "d /run/samba 0755 root root -" > $(DESTDIR)/usr/lib/tmpfiles.d/samba.conf
+	echo "d /run/ctdb 0755 root root -" > $(DESTDIR)/usr/lib/tmpfiles.d/ctdb.conf
 
 override_dh_installdocs-arch:
 	cp ctdb/config/events/README ctdb/README.eventscripts
@@ -181,6 +183,16 @@ ifeq ($(DEB_HOST_ARCH_OS), kfreebsd)
 	dh_installdocs -pctdb debian/ctdb.README.kfreebsd
 endif
 
+override_dh_installexamples-arch:
+	# CTDB config examples and helper scripts
+	mkdir -p ctdb/doc/examples/nfs-kernel-server
+	cp debian/ctdb.example.enable.nfs.sh ctdb/doc/examples/nfs-kernel-server/enable-nfs.sh
+	cp debian/ctdb.example.nfs-common ctdb/doc/examples/nfs-kernel-server/nfs-common
+	cp debian/ctdb.example.nfs-kernel-server ctdb/doc/examples/nfs-kernel-server/nfs-kernel-server
+	cp debian/ctdb.example.services ctdb/doc/examples/nfs-kernel-server/services
+	cp debian/ctdb.example.sysctl-nfs-static-ports.conf ctdb/doc/examples/nfs-kernel-server/99-nfs-static-ports.conf
+	dh_installexamples
+
 override_dh_installchangelogs:
 	dh_installchangelogs
 
@@ -267,3 +279,4 @@ override_dh_auto_clean:
 	rm -f .lock-wscript
 	rm -f ctdb/README.eventscripts
 	rm -f ctdb/README.notification
+	rm -rf ctdb/doc/examples/nfs-kernel-server/