Merge ~ubuntu-docker-images/ubuntu-docker-images/+git/telegraf:20.04-resync-with-21.04-dockerfile into ~ubuntu-docker-images/ubuntu-docker-images/+git/telegraf:edge

[Bryce Harrington]

This is a partial sync-up of the telegraf 20.04 branch to the changes on 21.04.

The build process for telegraf on 20.04 is substantially different from 21.04, so out of caution rather than copy the 21.04 Dockerfile and fixup, I manually made the corresponding refactoring changes but left the existing build system essentially as-is. Let me know if this should be done differently.

A few other things to call out. The 20.04 branch did not have a manifest, so I added the basic support there. However, the 21.04 image also had logic to include go packages. Is that needed here as well? I'm guessing the new golang-go and/or python dependencies were added for this?

In the 20.04 Dockerfile there is a "COPY . ." line that isn't in 21.04. I don't know what that does so left it in place, but can it be removed?

In building the image, I got an error regarding 'apt-get install ... /pkg/telegraf*.deb ...' due to "unsupported file given". Instead of using apt-get to install the .debs I used dpkg -i, and that seemed to resolve the issue.

With that change, I verified the branch builds the docker image successfully, locally. I'll appreciate your usual checking that it builds and runs ok for you too.

For ease of review of the remaining delta, here is a diff between the two Dockerfiles after my changes (and ignoring whitespace changes):

    https://paste.ubuntu.com/p/wKHFBMBJ6g/

[Bryce Harrington (bryce)]

(Refreshed the pastebin)

[Sergio Durigan Junior (sergiodj)]

Thanks for the MP, Bryce.

I am leaving a few comments here and there for small nits I found. I built and tested the image, and everything seems OK (modulo the manifest thing; see below).

For 21.04, since we have telegraf in the archive, the build process is much simpler because it's just a matter of apt-get installing the package. For 20.04, the situation is a bit more complex. We are still working with the SRU team in order to include telegraf into Focal, but it's not there yet. However, we do have the (yet to be) Focal telegraf package in a repository, so that's what the Dockerfile uses to build things (we could perhaps maintain this in a PPA, but I decided to just build everything from source). That's why you see the differences between the two releases.

For the manifest generation, as I mentioned below, I can look into it if you'd like. It's not entirely trivial and I already did it for 21.04, so it will be faster, I think. You can leave the existing hunk that generates the manifest from dpkg, because that will be useful as well.

I'm approving this, so after you fix the issues I pointed, feel free to merge it. Thanks.

review: Approve

[Bryce Harrington (bryce)]

On Wed, Mar 24, 2021 at 10:47:12PM -0000, Sergio Durigan Junior wrote:
> Review: Approve
>
> Thanks for the MP, Bryce.
>
> I am leaving a few comments here and there for small nits I found. I built and tested the image, and everything seems OK (modulo the manifest thing; see below).
>
> For 21.04, since we have telegraf in the archive, the build process is much simpler because it's just a matter of apt-get installing the package. For 20.04, the situation is a bit more complex. We are still working with the SRU team in order to include telegraf into Focal, but it's not there yet. However, we do have the (yet to be) Focal telegraf package in a repository, so that's what the Dockerfile uses to build things (we could perhaps maintain this in a PPA, but I decided to just build everything from source). That's why you see the differences between the two releases.
>
> For the manifest generation, as I mentioned below, I can look into it if you'd like. It's not entirely trivial and I already did it for 21.04, so it will be faster, I think. You can leave the existing hunk that generates the manifest from dpkg, because that will be useful as well.
>

Thanks, that'd be great. I pushed it as is, you can fix up at your leisure.

> I'm approving this, so after you fix the issues I pointed, feel free to merge it. Thanks.
>
> Diff comments:
>
> > diff --git a/Dockerfile b/Dockerfile
> > index 97b39b7..b99f9b1 100644
> > --- a/Dockerfile
> > +++ b/Dockerfile
> > @@ -1,37 +1,47 @@
> > -FROM ubuntu:focal AS builder
> > +FROM ubuntu:focal AS manifest-builder
>
> The name should remain "builder", since it's being used to build the full package, not only the manifest.

Done

> > COPY . .
> >
> > WORKDIR /src/telegraf
> >
> > -RUN apt-get update \
> > - && DEBIAN_FRONTEND=noninteractive apt-get upgrade -y \
> > - && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends git dpkg-dev ca-certificates pristine-tar \
> > - && git clone https://git.launchpad.net/~ubuntu-server/ubuntu/+source/telegraf \
> > - && cd telegraf \
> > - && git branch pristine-tar origin/pristine-tar \
> > - && git checkout focal \
> > - && cp -a /adjust-telegraf-conf.patch debian/patches \
> > - && echo "adjust-telegraf-conf.patch" >> debian/patches/series \
> > - && sed -i 's@^EXCLUDE_TESTS.*@EXCLUDE_TESTS = github.com/influxdata/telegraf/plugins/inputs/http_response \
> > +RUN set -eux; \
> > + apt-get update; \
> > + DEBIAN_FRONTEND=noninteractive apt-get full-upgrade -y; \
> > + DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
> > + dpkg-dev git ca-certificates pristine-tar; \
> > + git clone https://git.launchpad.net/~ubuntu-server/ubuntu/+source/telegraf; \
> > + cd telegraf; \
> > + git branch pristine-tar origin/pristine-tar; \
> > + git checkout focal; \
> > + cp -a /adjust-telegraf-conf.patch debian/patches/; \
> > + echo "adjust-telegraf-conf.patch" >> debian/patches/series; \
> > + sed -i 's@^EXCLUDE_TESTS.*@EXCLUDE_TESTS = github.com/influxdata/telegraf/plugins/inputs/http_response \
> > github.com/influxdata/telegraf/plugins/inputs/zipkin \
> > -github.com/influxdata/telegraf/plugins/outputs/prometheus_client ...

diff --git a/Dockerfile b/Dockerfile
index 97b39b7..b99f9b1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,37 +1,47 @@
-FROM ubuntu:focal AS builder
+FROM ubuntu:focal AS manifest-builder
 
 COPY . .
 
 WORKDIR /src/telegraf
 
-RUN apt-get update \
-	&& DEBIAN_FRONTEND=noninteractive apt-get upgrade -y \
-	&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends git dpkg-dev ca-certificates pristine-tar \
-	&& git clone https://git.launchpad.net/~ubuntu-server/ubuntu/+source/telegraf \
-	&& cd telegraf \
-	&& git branch pristine-tar origin/pristine-tar \
-	&& git checkout focal \
-	&& cp -a /adjust-telegraf-conf.patch debian/patches \
-	&& echo "adjust-telegraf-conf.patch" >> debian/patches/series \
-	&& sed -i 's@^EXCLUDE_TESTS.*@EXCLUDE_TESTS = github.com/influxdata/telegraf/plugins/inputs/http_response \
+RUN set -eux; \
+	apt-get update; \
+	DEBIAN_FRONTEND=noninteractive apt-get full-upgrade -y; \
+	DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+		dpkg-dev git ca-certificates pristine-tar; \
+	git clone https://git.launchpad.net/~ubuntu-server/ubuntu/+source/telegraf; \
+	cd telegraf; \
+	git branch pristine-tar origin/pristine-tar; \
+	git checkout focal; \
+	cp -a /adjust-telegraf-conf.patch debian/patches/; \
+	echo "adjust-telegraf-conf.patch" >> debian/patches/series; \
+	sed -i 's@^EXCLUDE_TESTS.*@EXCLUDE_TESTS = github.com/influxdata/telegraf/plugins/inputs/http_response \
 github.com/influxdata/telegraf/plugins/inputs/zipkin \
-github.com/influxdata/telegraf/plugins/outputs/prometheus_client \\@' debian/rules \
-	&& pristine-tar checkout ../telegraf_$(dpkg-parsechangelog -SVersion | sed 's/\(.*\)-.ubuntu.*/\1/').orig.tar.xz \
-	&& DEBIAN_FRONTEND=noninteractive apt-get build-dep . -y \
-	&& dpkg-buildpackage -us -uc
+github.com/influxdata/telegraf/plugins/outputs/prometheus_client \\@' debian/rules; \
+	pristine-tar checkout ../telegraf_$(dpkg-parsechangelog -SVersion | sed 's/\(.*\)-.ubuntu.*/\1/').orig.tar.xz; \
+	DEBIAN_FRONTEND=noninteractive apt-get build-dep . -y; \
+	dpkg-buildpackage -us -uc
 
 FROM ubuntu:focal
 
 ENV TZ UTC
 
-COPY --from=builder /src/telegraf/telegraf*.deb /pkg/
+COPY --from=manifest-builder /src/telegraf/telegraf*.deb /pkg/
 
-RUN apt-get update \
-	&& DEBIAN_FRONTEND=noninteractive apt-get upgrade -y \
-	&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends /pkg/telegraf*.deb tzdata \
+RUN set -eux; \
+	apt-get update; \
+	DEBIAN_FRONTEND=noninteractive apt-get full-upgrade -y; \
+	DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+		tzdata; \
+	dpkg -i /pkg/telegraf*.deb; \
 # We don't need to keep the .deb packages around
-	&& rm -rf /pkg \
-	&& rm -rf /var/lib/apt/lists/*
+	rm -rf /pkg; \
+	rm -rf /var/lib/apt/lists/*; \
+# smoke test
+    telegraf --version; \
+# create manifest
+    mkdir -p /usr/share/rocks; \
+    (echo "# os-release" && cat /etc/os-release && echo "# dpkg-query" && dpkg-query -f '${db:Status-Abbrev},${binary:Package},${Version},${source:Package},${Source:Version}\n' -W) > /usr/share/rocks/dpkg.query
 
 EXPOSE     9273
 ENTRYPOINT [ "/usr/bin/telegraf" ]