Merge ~chad.smith/ubuntu/+source/pollinate:ubuntu/trusty-devel into ubuntu/+source/pollinate:ubuntu/trusty-devel

[Chad Smith]

Upstream snapshot of pollinate 4.33 into trusty devel. No packaging behavior changes made.

[Scott Moser (smoser)]

We're missing the ChangeLog changes that came from upstream.

Then, for the debian/changelog new entries i had a comment below. Yours is fine for sure.

[Scott Moser (smoser)]

Filed and fixed bug 1774043.

I'm OK with either option here:
a.) new upstream release of 4.33 and upload to cosmic, then SRU that
b.) cherry-pick the patch to debian/patches/ for trusty.

[Scott Moser (smoser)]

I always kind of struggle for what a SRU changelog
should look like. If someone is using trusty and is reading changelogs
then they're interested in changes that they see. So the
'include img/build_name/server in user-agent' is not speficially new
as it is covered by 'Add cloud image build info'.

The same is true of 'fix bug on xen....' that never existed in trusty.
Obviously writing such a changelog means different changelogs for
each release we're doing an SRU to, and is generally more complex for
you. But does seem to me at least to provide more value to the user.

An example of a trusty and xenial changelog might be.
 * New upstream release (LP: #1761240):
   - Add hypervisor to useragent string
   - Add cloud image build info and data in /etc/pollinate/add-user-agent
     to useragent string.
   - Add '--print-user-agent' flag.
   - Speedups when collecting information for user-agent.
   - Get logger version via logger -V rather than dpkg query.
   - Perform safe read of /proc/uptime

Artful would be the same except for the 'Perform safe read..'
and bionic would have:
  - Use systemd-detect-virt if available.
  - Fix bug on xen when not booted with systemd.
  - Include img/build_name/server in user-agent.

fea7205... by Chad Smith on 2018-05-30

Import version 4.33-0ubuntu1~14.04.1 to ubuntu/trusty-proposed

New changelog entries:
  * New upstream release (LP: #1761240):
    - Add '--print-user-agent' flag.
    - Speedups when collecting information for user-agent.
    - Add hypervisor to useragent string
    - Get logger version via logger -V rather than dpkg query.
    - Add cloud image build info and data in /etc/pollinate/add-user-agent
      to user-agent string.
    - Perform as safe read of /proc/uptime

[Scott Moser (smoser)]

I made a small change here, to the changelog, just to clean up english:
- - Perform as safe read of /proc/uptime
+ - Perform a safe read of /proc/uptime

And then uploaded and changed trusty-updates to just trusty.

https://code.launchpad.net/~smoser/ubuntu/+source/pollinate/+git/pollinate/+ref/bug/1761240-sru-trusty

review: Approve

[Scott Moser (smoser)]

marking merged as uploaded to queue.

diff --git a/ChangeLog b/ChangeLog
index b0fe4c3..40a1039 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,4 +1,86 @@
-pollinate (4.25) released; urgency=medium
+pollinate (4.33) released; urgency=medium
+
+  * pollinate: fix bug on xen when not booted with systemd. user-agent
+    would contain virt/virt/xen rather than virt/xen. (LP: #1774043)
+  * pollinate: use systemd-detect-virt if available rather than detecting
+    its availability based on /run/systemd.
+
+ -- Scott Moser <[e-mail]>  Fri, 25 May 2018 12:30:15 -0400
+
+pollinate (4.32-0ubuntu1) cosmic; urgency=medium
+
+  * pollinate: include img/build_name/server in user-agent.
+
+ -- Scott Moser <[e-mail]>  Fri, 25 May 2018 12:30:06 -0400
+
+pollinate (4.31-0ubuntu1) bionic; urgency=medium
+
+  [ Scott Moser ]
+  * pollinate: add '--print-user-agent' flag.
+  * pollinate: Speedups when collecting information for user-agent.
+
+ -- Dustin Kirkland <[e-mail]>  Wed, 14 Feb 2018 11:01:18 -0600
+
+pollinate (4.30-0ubuntu1) bionic; urgency=medium
+
+  * debian/pollinate.postinst:
+    - don't try to chown a dir to the user, potentially before the user exists
+
+ -- Dustin Kirkland <[e-mail]>  Wed, 31 Jan 2018 07:24:33 -0600
+
+pollinate (4.29-0ubuntu1) bionic; urgency=medium
+
+  * No change rebuild, release error in 4.28
+
+ -- Dustin Kirkland <[e-mail]>  Tue, 30 Jan 2018 16:41:13 -0600
+
+pollinate (4.28-0ubuntu1) bionic; urgency=medium
+
+  [ Dustin Kirkland ]
+  * pollinate:
+    - add hypervisor to useragent string
+  * debian/pollinate.postinst:
+    - ensure that the pollinate user owns /var/cache/pollinate;
+      to be safe, we're going to do this just after the mkdir, and
+      after the adduser;
+      this avoids the warning:
+      + adduser: Warning: The home directory /var/cache/pollinate
+        does not belong to the user you are currently creating.
+
+  [ Steve Langasek ]
+  * Minor enhancements to pollinate runtime:
+    - dpkg | awk is unnecessary, dpkg-query --showformat does this
+  * dpkg -l is a fairly heavyweight operation (.1s); ask logger directly
+    what its version number is (.01s)
+
+  [ Scott Moser ]
+  * pollinate:
+    - Add cloud image build info and data in /etc/pollinate/add-user-agent.
+    - Use systemd-detect-virt and namespace virt/<type> in useragent
+
+ -- Dustin Kirkland <[e-mail]>  Tue, 30 Jan 2018 16:31:24 -0600
+
+pollinate (4.27-0ubuntu1) artful; urgency=medium
+
+  * Fixup maintscript & version missmatch from the previous upload.
+
+ -- Dimitri John Ledkov <[e-mail]>  Fri, 18 Aug 2017 13:13:16 +0100
+
+pollinate (4.26-0ubuntu1) artful; urgency=medium
+
+  [ Scott Moser ]
+  * Perform as safe read of /proc/uptime LP: #1708461
+
+  [ Dimitri John Ledkov ]
+  * Drop upstart system job
+  * Drop no longer needed build-deps, and redundantly specified essential
+    bsdutils
+  * Fix spelling typo in the description
+  * Bump debhelper compat to 9, previous levels are deprecated
+
+ -- Dustin Kirkland <[e-mail]>  Sat, 14 Jan 2017 08:45:46 +0100
+
+pollinate (4.25-0ubuntu1) zesty; urgency=medium
 
   [ Stefano Rivera ]
   * debian/control: Add missing dependency on xxd. (LP: #1656484)
@@ -6,7 +88,7 @@ pollinate (4.25) released; urgency=medium
   [ Dustin Kirkland ]
   * debian/control: xxd is provided by vim-common in older Ubuntu releases
 
- -- Dustin Kirkland <[e-mail]>  Wed, 02 Nov 2016 14:47:01 +0200
+ -- Dustin Kirkland <[e-mail]>  Sat, 14 Jan 2017 08:45:44 +0100
 
 pollinate (4.24-0ubuntu1) zesty; urgency=medium
 
diff --git a/debian/changelog b/debian/changelog
index 48eb078..0220678 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,16 @@
+pollinate (4.33-0ubuntu1~14.04.1) trusty-updates; urgency=medium
+
+  * New upstream release (LP: #1761240):
+    - Add '--print-user-agent' flag.
+    - Speedups when collecting information for user-agent.
+    - Add hypervisor to useragent string
+    - Get logger version via logger -V rather than dpkg query.
+    - Add cloud image build info and data in /etc/pollinate/add-user-agent
+      to user-agent string.
+    - Perform as safe read of /proc/uptime
+
+ -- Chad Smith <[e-mail]>  Wed, 30 May 2018 10:22:11 -0600
+
 pollinate (4.25-0ubuntu1~14.04.1) trusty; urgency=medium
 
   * New upstream release (LP: #1708192):
diff --git a/pollinate b/pollinate
index a69cd9f..4a4cbbd 100755
--- a/pollinate
+++ b/pollinate
@@ -17,6 +17,7 @@
 #  along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 set -e
+set -f
 
 PKG="pollinate"
 TMPDIR=$(mktemp -d -t "${PKG}.XXXXXXXXXXXX")
@@ -28,8 +29,8 @@ HOSTNAME=$(hostname)
 STRICT=0
 
 # Only recent logger version supports --id=[ID]
-logger_ver=$(dpkg -l bsdutils | tail -n1 | awk '{print $3}')
-dpkg --compare-versions $logger_ver ge 1:2.26.2 && LOGGER="logger --id=$$" || LOGGER="logger"
+logger_ver=$(logger -V 2>&1 | awk '{print $4}')
+dpkg --compare-versions $logger_ver ge 2.26.2 && LOGGER="logger --id=$$" || LOGGER="logger"
 # Log to both syslog, and stderr, if we're on an interactive terminal
 [ -t 0 ] && LOGGER="$LOGGER -s"
 
@@ -88,20 +89,121 @@ hash_and_write() {
 	log "client hashed response from [${1}]"
 }
 
+read_build_info() {
+	# ubuntu images place build information in /etc/cloud/build.info
+	# format of file is '<key>: <value>' put these under img/<key>/<value>
+	local bifile="${1:-/etc/cloud/build.info}" ret=""
+	_RET=""
+	[ -s "$bifile" ] || return 0
+	ret=$(awk '{
+		gsub(/#.*/, ""); gsub(/\s+$/, "");
+		if ($0 == "" || $0 !~ /:/) next;
+		gsub(/:\s*/, "/");
+		printf("img/%s ", $0) }' "$bifile") || return
+	_RET="${ret% }"
+}
+
+read_addl_info() {
+	# allow additinal info file to contain entries one per line.  lines must
+	# have a '/' in them. remove trailing space and '#' as comment. example:
+	#   key/value
+	#   fookey/foovalue # written by foo
+	local aifile="${1:-/etc/pollinate/add-user-agent}" ret=""
+	_RET=""
+	[ -s "$aifile" ] || return 0
+	ret=$(awk '{
+		gsub(/#.*/, ""); gsub(/\s+$/, "");
+		if ($0 == "" || $0 !~ /\//) next;
+		printf("%s ", $0); }' "$aifile") || return
+	_RET=${ret% }
+}
+
+read_virt() {
+	# return virt/<value> where value is the virtualization platform the
+	# system is running on.
+	local ret=""
+	_RET=""
+	if command -v systemd-detect-virt >/dev/null; then
+		ret=$(systemd-detect-virt)
+		# systemd-detect-virt returns 1 for 'none'
+		[ $? -eq 0 -o "$ret" = "none" ] || ret=""
+	else
+		# trusty would take this path.
+		if [ -d /dev/xen ]; then
+			ret="xen"
+		elif [ -d /dev/lxd ]; then
+			# call this 'lxc' for consistency with systemd-detect-virt.
+			ret="lxc"
+		elif dmesg | grep --quiet " kvm-clock:"; then
+			ret="kvm"
+		fi
+	fi
+	[ -n "$ret" ] || return
+	_RET="virt/$ret"
+}
+
+read_package_versions() {
+	local pkgs="pollinate curl cloud-init" data="" p=""
+	data=$(dpkg-query \
+		-W --showformat='${Package}/${Version} ' $pkgs 2>/dev/null) || :
+	# fill in 'package/' for any package not installed.
+	for p in ${pkgs}; do
+		[ "${data#*$p/}" = "$data" ] && data="${data} $p/"
+	done
+	if [ -n "$TESTING" ]; then
+		p="pollinate"
+		data="${data%%$p/*}$p${TESTING}/${data#*$p/}"
+	fi
+	set -- $data
+	_RET="$*"
+}
+
+read_uname_info() {
+	# taken from cloud-init ds-identify.
+	# run uname, and parse output.
+	# uname is tricky to parse as it outputs always in a given order
+	# independent of option order. kernel-version is known to have spaces.
+	# 1   -s kernel-name
+	# 2   -n nodename
+	# 3   -r kernel-release
+	# 4.. -v kernel-version(whitespace)
+	# N-2 -m machine
+	# N-1 -o operating-system
+	local out="" krel="" machine="" os=""
+	out=$(uname -snrvmo) || { _RET=""; return; }
+	set -- $out
+	krel="$3"
+	shift 3
+	while [ $# -gt 2 ]; do
+		shift
+	done
+	machine=$1
+	os=$2
+	_RET="$os/$krel/$machine"
+	return 0
+}
+
 user_agent() {
 	# Construct a user agent, with useful debug information
 	# Very similar to Firefox and Chrome
-	local pollinate_ver="$(dpkg -l pollinate | awk '$1 == "ii" { print($3); exit(0); }')"
-	local curl_ver="$(dpkg -l curl | awk '$1 == "ii" { print($3); exit(0); }')"
-	local cloud_init_ver="$(dpkg -l cloud-init 2>/dev/null | awk '$1 == "ii" { print($3); exit(0); }')"
+	
 	. /etc/lsb-release
-	local lsb=$(echo "${DISTRIB_DESCRIPTION}" | sed -e "s/ /\//g")
-	local platform="$(uname -o)/$(uname -r)/$(uname -m)"
-	local cpu="$(grep -m1 "^model name" /proc/cpuinfo | sed -e "s/.*: //" -e "s:\s\+:/:g")"
-	local up idle
-	read up idle < /proc/uptime
-	local uptime="uptime/$up/$idle"
-	USER_AGENT="pollinate${TESTING}/${pollinate_ver} curl/${curl_ver} cloud-init/${cloud_init_ver} ${lsb} ${platform} ${cpu} ${uptime}"
+
+	local pkg_info="" lsb="" platform="" cpu="" up="NA" idle="NA" uptime
+	read_package_versions && pkg_info="$_RET"
+	read_uname_info && platform="$_RET"
+
+	lsb=$(echo "${DISTRIB_DESCRIPTION}" | sed -e "s/ /\//g")
+	cpu="$(grep -m1 "^model name" /proc/cpuinfo | sed -e "s/.*: //" -e "s:\s\+:/:g")"
+	{ read up idle < /proc/uptime ; } >/dev/null 2>&1 || :
+	uptime="uptime/$up/$idle"
+
+	local addl_data="" build_info="" virt=""
+	read_build_info && build_info="${_RET}"
+	read_addl_info && addl_data="${_RET}"
+	read_virt && virt="${_RET}"
+
+	USER_AGENT="${pkg_info} ${lsb} ${platform} ${cpu} ${uptime}${virt:+ ${virt}}${build_info:+ ${build_info}}${addl_data:+ ${addl_data}}"
 }
 
 exchange() {
@@ -203,6 +305,11 @@ while [ ! -z "$1" ]; do
 			WAIT="$2"
 			shift 2
 		;;
+		--print-user-agent)
+			user_agent || error "Failed to get user-agent."
+			echo "${USER_AGENT}"
+			exit
+		;;
 		*)
 			error "Unknown options [$1]"
 		;;
diff --git a/pollinate.1 b/pollinate.1
index f7890ee..8e2b9ce 100644
--- a/pollinate.1
+++ b/pollinate.1
@@ -45,6 +45,9 @@ silence any log messages
 .TP
 \fB-w|--wait\fP WAIT
 time in seconds you're willing to wait for a response from the server, uses \fBtimeout\fP(1)
+.TP
+\fB   --print-user-agent\fP
+Print the user-agent that is used and exit immediately.
 
 .SH SEE ALSO
 \fBpollen\fP(8), \fBtimeout\fP(1)