Merge ~ahasenack/ubuntu/+source/haproxy:disco-haproxy-better-dep8 into ubuntu/+source/haproxy:ubuntu/devel

[Andreas Hasenack]

Bug #1804069 allowed us to ship a broken haproxy on arm64, and a test as simple as this would have caught it.

This branch adds a simple, yet better, DEP8 test to haproxy. I came up with it after writing the testing instructions for that bug, and it's basically a copy of that.

Bileto ticket: https://bileto.ubuntu.com/#/ticket/3613 (still running, I'll keep an eye on it)

I'm inclined to add this test to the #1804069 SRU. Let's see how reviews go, and how it fares in bileto. I can then also submit this to Debian.

Here is a good local run:
...
autopkgtest [18:42:01]: test proxy-localhost: [-----------------------
+ cat
+ systemctl restart haproxy
+ cd /var/www/html
+ md5sum index.html
+ src_md5=3526531ccd6c6a1d2340574a305a18f8 index.html
+ cd -
+ rm -f index.html
/tmp/autopkgtest.oDV55n/build.aj4/real-tree
+ wget -t1 http://localhost:8080 -O index.html
--2019-01-24 20:42:01-- http://localhost:8080/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10918 (11K) [text/html]
Saving to: ‘index.html’

     0K .......... 100% 159M=0s

2019-01-24 20:42:01 (159 MB/s) - ‘index.html’ saved [10918/10918]

+ result=0
+ echo 3526531ccd6c6a1d2340574a305a18f8 index.html
+ md5sum -c
index.html: OK
+ [ 0 != 0 ]
+ echo OK: index.html downloaded via haproxy matches the source file.
+ exit 0
OK: index.html downloaded via haproxy matches the source file.
autopkgtest [18:42:02]: test proxy-localhost: -----------------------]
autopkgtest [18:42:02]: test proxy-localhost: - - - - - - - - - - results - - - - - - - - - -
proxy-localhost PASS
autopkgtest [18:42:02]: @@@@@@@@@@@@@@@@@@@@ summary
cli PASS
proxy-localhost PASS

[Andreas Hasenack (ahasenack)]

While bileto is being fixed, these are the (green) results for this package:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_39a8dbb93caf4ec889f8a1b7f69885db/bileto-3613-excuses/2019-01-29_10:20:01/3613_disco_excuses.html

[Christian Ehrhardt  (paelzer)]

Generally a simple but quite useful test, thanks for adding that.
I'm nit-picking and leave it to you what you want to fix - it is "good enough" to go as-is if that is what you prefer.

On the question to add the test on SRU.
IMHO the tradeoff usually is:
- do not add flaky tests (this one seems stable)
- do not SRU for just a test as we trigger many downloads around the world (here we SRU the fix anyway)
=> Therefore if it is not too much effort (e.g. the config would be vastly different on older versions) I'd lean to adding the test on the SRUs as well.

review: Needs Information

[Andreas Hasenack (ahasenack)]

This new DEP8 test would actually check that the bug the SRU is fixing is really fixed. The SRU itself will happen, I would just piggy back this new test on it, and it's a test related to the SRU.

[Andreas Hasenack (ahasenack)]

Responded inline

[Christian Ehrhardt  (paelzer)]

With the improved subshell (with sed to strip the path instead of cd) it LGTM.
+1

review: Approve

[Andreas Hasenack (ahasenack)]

New run, using sed:
autopkgtest [11:37:00]: test proxy-localhost: [-----------------------
+ cat
+ systemctl restart haproxy
+ + md5sum /var/www/html/index.html
sed s,/var/www/html/,,
+ src_md5=3526531ccd6c6a1d2340574a305a18f8 index.html
+ rm -f index.html
+ wget -t1 http://localhost:8080 -O index.html
--2019-01-29 13:37:00-- http://localhost:8080/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10918 (11K) [text/html]
Saving to: ‘index.html’

     0K .......... 100% 197M=0s

2019-01-29 13:37:00 (197 MB/s) - ‘index.html’ saved [10918/10918]

+ result=0
+ echo 3526531ccd6c6a1d2340574a305a18f8 index.html
+ md5sum -c
+ [ 0 != 0 ]
index.html: OK
+ echo OK: index.html downloaded via haproxy matches the source file.
+ OK: index.html downloaded via haproxy matches the source file.
exit 0
autopkgtest [11:37:00]: test proxy-localhost: -----------------------]
proxy-localhost PASS
autopkgtest [11:37:01]: test proxy-localhost: - - - - - - - - - - results - - - - - - - - - -
autopkgtest [11:37:01]: @@@@@@@@@@@@@@@@@@@@ summary
cli PASS
proxy-localhost PASS

[Andreas Hasenack (ahasenack)]

Salsa MP: https://salsa.debian.org/haproxy-team/haproxy/merge_requests/3

[Andreas Hasenack (ahasenack)]

I switched to using cmp, as suggested by rbasak in the salsa MP above. New output in ubuntu:

autopkgtest [15:19:20]: test proxy-localhost: [-----------------------
+ cat
+ systemctl restart haproxy
+ wget -t1 http://localhost:8080 -O-
+ cmp /var/www/html/index.html -
--2019-01-29 17:19:21-- http://localhost:8080/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10918 (11K) [text/html]
Saving to: ‘STDOUT’

     0K .......... 100% 396M=0s

2019-01-29 17:19:21 (396 MB/s) - written to stdout [10918/10918]

+ echo OK: index.html downloaded via haproxy matches the source file.
+ exit 0
OK: index.html downloaded via haproxy matches the source file.
autopkgtest [15:19:21]: test proxy-localhost: -----------------------]
autopkgtest [15:19:21]: test proxy-localhost: - - - - - - - - - - results - - - - - - - - - -
proxy-localhost PASS
autopkgtest [15:19:21]: @@@@@@@@@@@@@@@@@@@@ summary
cli PASS
proxy-localhost PASS

[Robie Basak (racb)]

review: Approve

[Andreas Hasenack (ahasenack)]

dep8 is green with the latest changes, preparing upload.

[Andreas Hasenack (ahasenack)]

Tagged and uploaded.
$ git push pkg upload/1.8.17-1ubuntu1
Enumerating objects: 21, done.
Counting objects: 100% (21/21), done.
Delta compression using up to 4 threads
Compressing objects: 100% (12/12), done.
Writing objects: 100% (16/16), 2.25 KiB | 2.25 MiB/s, done.
Total 16 (delta 9), reused 9 (delta 4)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/haproxy
 * [new tag] upload/1.8.17-1ubuntu1 -> upload/1.8.17-1ubuntu1

$ dput ubuntu ../haproxy_1.8.17-1ubuntu1_source.changes
Checking signature on .changes
gpg: ../haproxy_1.8.17-1ubuntu1_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../haproxy_1.8.17-1ubuntu1.dsc: Valid signature from AC983EB5BF6BCBA9
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading haproxy_1.8.17-1ubuntu1.dsc: done.
  Uploading haproxy_1.8.17-1ubuntu1.debian.tar.xz: done.
  Uploading haproxy_1.8.17-1ubuntu1_source.buildinfo: done.
  Uploading haproxy_1.8.17-1ubuntu1_source.changes: done.
Successfully uploaded packages.

[Christian Ehrhardt  (paelzer)]

Yeah looks even better with cmp.

diff --git a/debian/changelog b/debian/changelog
index de32221..3f24b6e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+haproxy (1.8.17-1ubuntu1) disco; urgency=medium
+
+  * d/t/control, d/t/proxy-localhost: simple DEP8 test to actually
+    generate traffic through haproxy.
+
+ -- Andreas Hasenack <[e-mail]>  Thu, 24 Jan 2019 18:11:39 -0200
+
 haproxy (1.8.17-1) unstable; urgency=medium
 
   * New upstream version 1.8.17
diff --git a/debian/tests/control b/debian/tests/control
index 13837c2..70649e3 100644
--- a/debian/tests/control
+++ b/debian/tests/control
@@ -1,3 +1,7 @@
 Tests: cli
 Depends: haproxy, socat
 Restrictions: needs-root
+
+Tests: proxy-localhost
+Depends: haproxy, wget, apache2
+Restrictions: needs-root, allow-stderr, isolation-container
diff --git a/debian/tests/proxy-localhost b/debian/tests/proxy-localhost
new file mode 100644
index 0000000..2824287
--- /dev/null
+++ b/debian/tests/proxy-localhost
@@ -0,0 +1,48 @@
+#!/bin/sh
+
+set -eux
+
+cat > /etc/haproxy/haproxy.cfg <<EOF
+global
+        chroot /var/lib/haproxy
+        user haproxy
+        group haproxy
+        daemon
+        maxconn 4096
+
+defaults
+        log global
+        option dontlognull
+        option redispatch
+        retries 3
+        timeout client 50s
+        timeout connect 10s
+        timeout http-request 5s
+        timeout server 50s
+        maxconn 4096
+
+frontend test-front
+    bind *:8080
+    mode http
+    default_backend test-back
+
+backend test-back
+    mode http
+    stick store-request src
+    stick-table type ip size 256k expire 30m
+    server test-1 localhost:80
+EOF
+
+systemctl restart haproxy
+
+# index.html is shipped with apache2
+# Download it via haproxy and compare
+if wget -t1 http://localhost:8080 -O- | cmp /var/www/html/index.html -; then
+    echo "OK: index.html downloaded via haproxy matches the source file."
+else
+    echo "FAIL: downloaded index.html via haproxy is different from the"
+    echo "      file delivered by apache."
+    exit 1
+fi
+
+exit 0