Ubuntu
ec2-hibinit-agent package

Merge ~mitchdz/ubuntu/+source/ec2-hibinit-agent:mitch/xenial-imdsv2 into ubuntu/+source/ec2-hibinit-agent:ubuntu/xenial-devel

  1. Git
  2. lp:~mitchdz/ubuntu/+source/ec2-hibinit-agent
  3. mitch/xenial-imdsv2
  4. Merge into ubuntu/xenial-devel
Proposed by Mitchell Dzurick
Status: Merged
Merge reported by: Mitchell Dzurick
Merged at revision: 577603dc5c8a0c3bce9c6992f28532985a514c2e
Proposed branch: ~mitchdz/ubuntu/+source/ec2-hibinit-agent:mitch/xenial-imdsv2
Merge into: ubuntu/+source/ec2-hibinit-agent:ubuntu/xenial-devel
Diff against target: 375 lines (+312/-2)
8 files modified
debian/changelog (+16/-0)
debian/ec2-hibinit-agent.install (+1/-0)
debian/hibinit-agent.system-sleep (+1/-2)
debian/hibinit-resume (+48/-0)
debian/patches/lp1941785-Add-support-for-IMDSv2.patch (+161/-0)
debian/patches/lp1968805-Swapon-with-maximum-priority-before-hibernation.patch (+63/-0)
debian/patches/lp2023924-remove-quotes-from-state-dir.patch (+19/-0)
debian/patches/series (+3/-0)
Reviewer Review Type Date Requested Status
git-ubuntu import Pending
Review via email: mp+456027@code.launchpad.net
To post a comment you must log in.
~mitchdz/ubuntu/+source/ec2-hibinit-agent:mitch/xenial-imdsv2 updated
92c89f4... by Mitchell Dzurick

debian/hibinit-resume: Add extra steps around swapoff to avoid OOM errors.

4054ec8... by Mitchell Dzurick

d/p/lp1968805-Swapon-with-maximum-priority-before-hibernation.patch: Swapon with maximum priority right before hibernation.

13f7bf9... by Mitchell Dzurick

d/p/{lp1941785,lp2023924}*: add IMDSv2, fix quotes bug created from IMDSv2 patch

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/debian/changelog b/debian/changelog
index b366dca..b4f95a5 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,19 @@
1ec2-hibinit-agent (1.0.0-0ubuntu4~16.04.5) xenial; urgency=medium
2
3 * d/p/lp2023924-remove-quotes-from-state-dir.patch: fixes an issue where a
4 weird directory would be created in root (LP: #2023924)
5 * d/p/lp1941785-Add-support-for-IMDSv2.patch: allow hibernation of
6 AWS EC2 instances with IMDSv2 (LP: #1941785)
7 * Swapon with maximum priority right before hibernation. This resolves
8 swapfile priority issues with additional or multiple swapfiles enabled.
9 (LP: #1968805)
10 - d/p/lp1968805-Swapon-with-maximum-priority-before-hibernation.patch
11 * debian/hibinit-resume: Add extra steps around swapoff to avoid OOM errors.
12 Also work around xen-netfront not resuming properly.
13 Thanks to Francis Ginther for the initial patch (LP: #1863242, #1864041)
14
15 -- Mitchell Dzurick <mitchell.dzurick@canonical.com> Tue, 21 Nov 2023 15:31:14 -0700
16
1ec2-hibinit-agent (1.0.0-0ubuntu4~16.04.4) xenial; urgency=medium17ec2-hibinit-agent (1.0.0-0ubuntu4~16.04.4) xenial; urgency=medium
218
3 * Always set resume device by PARTUUID instead of by device name.19 * Always set resume device by PARTUUID instead of by device name.
diff --git a/debian/ec2-hibinit-agent.install b/debian/ec2-hibinit-agent.install
index b140d28..5b1b272 100755
--- a/debian/ec2-hibinit-agent.install
+++ b/debian/ec2-hibinit-agent.install
@@ -5,5 +5,6 @@ debian/ec2-hibinit-agent-ignore-powerkey.conf usr/lib/systemd/logind.conf.d/
5debian/hibinit-agent.system-sleep => lib/systemd/system-sleep/hibinit-agent5debian/hibinit-agent.system-sleep => lib/systemd/system-sleep/hibinit-agent
6debian/hibinit-power etc/acpi/events6debian/hibinit-power etc/acpi/events
7debian/hibinit-power.sh etc/acpi/actions7debian/hibinit-power.sh etc/acpi/actions
8debian/hibinit-resume usr/lib/ec2-hibinit-agent
8etc/hibinit-config.cfg9etc/hibinit-config.cfg
9agent/* usr/bin/10agent/* usr/bin/
diff --git a/debian/hibinit-agent.system-sleep b/debian/hibinit-agent.system-sleep
index 914161b..62db7bd 100755
--- a/debian/hibinit-agent.system-sleep
+++ b/debian/hibinit-agent.system-sleep
@@ -4,8 +4,7 @@ set -e
4if [ "$2" = "hibernate" ] || [ "$2" = "hybrid-sleep" ]; then4if [ "$2" = "hibernate" ] || [ "$2" = "hybrid-sleep" ]; then
5 case "$1" in5 case "$1" in
6 post)6 post)
7 systemctl restart --no-block systemd-networkd7 systemd-run --no-block --service-type=oneshot /usr/lib/ec2-hibinit-agent/hibinit-resume
8 systemd-run --no-block --service-type=oneshot swapoff /swap-hibinit
9 ;;8 ;;
10 esac9 esac
11fi10fi
diff --git a/debian/hibinit-resume b/debian/hibinit-resume
12new file mode 10075511new file mode 100755
index 0000000..e83fe2c
--- /dev/null
+++ b/debian/hibinit-resume
@@ -0,0 +1,48 @@
1#!/bin/bash
2#
3# Resume from hibernation on EC2 and apply workarounds for various issues
4#
5# Copyright (C) 2020 Canonical Ltd.
6#
7
8set -e
9
10# Starting hibinit resume actions
11echo "hibinit-resume: begin" >> /var/log/hibinit-resume.log
12
13# Give some time to make sure the system is fully up and running, see LP: #1864041
14sleep 2
15
16# Make sure the network is properly resumed
17echo "restarting network" >> /var/log/hibinit-resume.log
18if grep -q xen_netfront /proc/modules; then
19 # Xen netfront workaround LP: #1864041
20 modprobe -r xen_netfront
21 modprobe xen_netfront
22fi
23systemctl restart --no-block systemd-networkd
24
25# Temporarily set memory allocation policy to "always overcommit". With
26# the default overcommit policy some applications might experience
27# out-of-memory errors. Setting the policy to "always overcommit" seems
28# to be a safe way to prevent these errors.
29OVERCOMMIT_POLICY=$(cat /proc/sys/vm/overcommit_memory)
30echo 1 > /proc/sys/vm/overcommit_memory
31# Start swapoff and make sure swap file is gone
32while :; do
33 echo "starting swapoff" >> /var/log/hibinit-resume.log
34 swapoff /swap-hibinit
35 if grep -q swap-hibinit /proc/swaps; then
36 echo "swap still active" >> /var/log/hibinit-resume.log
37 sleep 10
38 else
39 break
40 fi
41done
42echo "swap disabled" >> /var/log/hibinit-resume.log
43cat /proc/swaps >> /var/log/hibinit-resume.log
44# Restore previou overcommit policy
45echo "$OVERCOMMIT_POLICY" > /proc/sys/vm/overcommit_memory
46
47# All done
48echo "hibinit-resume: completed " >> /var/log/hibinit-resume.log
diff --git a/debian/patches/lp1941785-Add-support-for-IMDSv2.patch b/debian/patches/lp1941785-Add-support-for-IMDSv2.patch
0new file mode 10064449new file mode 100644
index 0000000..6855eb2
--- /dev/null
+++ b/debian/patches/lp1941785-Add-support-for-IMDSv2.patch
@@ -0,0 +1,161 @@
1Description: Add support for IMDSv2
2Author: Frederick Lefebvre <fredlef@amazon.com>
3Origin: backport, https://github.com/aws/amazon-ec2-hibinit-agent/commit/9d9bca5c61fa9256289e68c88bd3747af2f62e28
4Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/ec2-hibinit-agent/+bug/1941785
5Last-Update: 2023-05-16
6---
7This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
8--- a/agent/hibinit-agent
9+++ b/agent/hibinit-agent
10@@ -18,6 +18,7 @@
11 import sys
12 import syslog
13 import math
14+import requests
15 from subprocess import check_call, check_output, STDOUT
16 from threading import Thread
17 from math import ceil
18@@ -25,11 +26,6 @@
19
20
21 try:
22- from urllib.request import urlopen, Request
23-except ImportError:
24- from urllib2 import urlopen, Request, HTTPError
25-
26-try:
27 from ConfigParser import ConfigParser, NoSectionError, NoOptionError
28 except:
29 from configparser import ConfigParser, NoSectionError, NoOptionError
30@@ -41,7 +37,12 @@
31 log_to_syslog = True
32 log_to_stderr = True
33 SWAP_FILE = '/swap-hibinit'
34-URL = "http://169.254.169.254/latest/meta-data/hibernation/configured"
35+
36+DEFAULT_STATE_DIR = '/var/lib/hibinit-agent'
37+HIB_ENABLED_FILE = "hibernation-enabled"
38+IMDS_BASEURL = 'http://169.254.169.254'
39+IMDS_API_TOKEN_PATH = 'latest/api/token'
40+IMDS_SPOT_ACTION_PATH = 'latest/meta-data/hibernation/configured'
41
42 def log(message):
43 if log_to_syslog:
44@@ -314,6 +315,9 @@
45 get_int('swap', 'percentage-of-ram'), args.swap_ram_percentage, 100)
46 self.swap_mb = self.merge(
47 get_int('swap', 'target-size-mb'), args.swap_target_size_mb, 4000)
48+ self.state_dir = get('core', 'state-dir')
49+ if self.state_dir is None:
50+ self.state_dir = DEFAULT_STATE_DIR
51
52
53 def merge(self, cf_value, arg_value, def_val):
54@@ -337,31 +341,55 @@
55 def __str__(self):
56 return str(self.__dict__)
57
58-def hibernationEnabled():
59- """Returns a boolean indicating whether hibernation is enabled or not."""
60- response = None
61- try:
62- response = urlopen(URL)
63- data = response.read()
64- if data.lower() in ('false', b'false'):
65- return False
66- except:
67- return False
68- finally:
69- if response:
70- response.close()
71- return True
72+def get_imds_token(seconds=21600):
73+ """ Get a token to access instance metadata. """
74+ log("Requesting new IMDSv2 token.")
75+ request_header = {'X-aws-ec2-metadata-token-ttl-seconds': str(seconds)}
76+ token_url = '{}/{}'.format(IMDS_BASEURL, IMDS_API_TOKEN_PATH)
77+ response = requests.put(token_url, headers=request_header)
78+ response.close()
79+ if response.status_code != 200:
80+ return None
81+
82+ return response.text
83+
84+def create_state_dir(state_dir):
85+ """ Create agent run dir if it doesn't exists."""
86+ if not os.path.isdir(state_dir):
87+ os.makedirs(state_dir)
88+
89+def hibernation_enabled(state_dir):
90+ """Returns a boolean indicating whether hibernation is enabled or not.
91+ Hibernation can't be enabled/disabled the instance launch. If we find
92+ hibernation to be enabled, we create a semephore file so that we don't
93+ have to probe IMDS again. That is useful when a instance is rebooted
94+ after/if the IMDS http endpoint has been disabled.
95+ """
96+ hib_sem_file = os.path.join(state_dir, HIB_ENABLED_FILE)
97+ if os.path.isfile(hib_sem_file):
98+ log("Found {!r}, configuring hibernation".format(hib_sem_file))
99+ return True
100+
101+ imds_token = get_imds_token()
102+ if imds_token is None:
103+ # IMDS http endpoint is disabled
104+ return False
105+
106+ request_header = {'X-aws-ec2-metadata-token': imds_token}
107+ response = requests.get("{}/{}".format(IMDS_BASEURL, IMDS_SPOT_ACTION_PATH),
108+ headers=request_header)
109+ response.close()
110+ if response.status_code != 200 or response.text.lower() == "false":
111+ return False
112+
113+ log("Hibernation Configured Flag found")
114+ os.mknod(hib_sem_file)
115+
116+ return True
117+
118
119 def main():
120
121- if not hibernationEnabled():
122- log("Instance Launch has not enabled Hibernation Configured Flag. hibinit-agent exiting!!")
123- exit(0)
124- # Validate if disk space>total RAM
125- ram_bytes = os.sysconf('SC_PAGE_SIZE') * os.sysconf('SC_PHYS_PAGES')
126- if get_rootfs_size()<=(math.ceil(float(ram_bytes)/(1024*1024*1024))):
127- log("Insufficient disk space. Cannot create setup for hibernation. Please allocate a larger root device")
128- exit(1)
129 # Parse arguments
130 parser = argparse.ArgumentParser(description="An EC2 background process that creates a setup for instance hibernation "
131 "at instance launch and also registers ACPI sleep event/actions")
132@@ -388,6 +416,17 @@
133 log_to_syslog = config.log_to_syslog
134
135 log("Effective config: %s" % config)
136+ create_state_dir(config.state_dir)
137+
138+ # Let's first check if we even need to run
139+ if not hibernation_enabled(config.state_dir):
140+ log("Instance Launch has not enabled Hibernation Configured Flag. hibinit-agent exiting!!")
141+ exit(0)
142+ # Validate if disk space>total RAM
143+ ram_bytes = os.sysconf('SC_PAGE_SIZE') * os.sysconf('SC_PHYS_PAGES')
144+ if get_rootfs_size()<=(math.ceil(float(ram_bytes)/(1024*1024*1024))):
145+ log("Insufficient disk space. Cannot create setup for hibernation. Please allocate a larger root device")
146+ exit(1)
147
148 target_swap_size = config.swap_mb * 1024 * 1024
149 swap_percentage_size = ram_bytes * config.swap_percentage // 100
150--- a/etc/hibinit-config.cfg
151+++ b/etc/hibinit-config.cfg
152@@ -11,6 +11,9 @@
153 # filesystems.
154 touch-swap = False
155
156+# Location where to create any state files
157+state-dir = "/var/lib/hibinit-agent"
158+
159 [swap]
160 # If there's no swap then we create it to be equal to the specified
161 # percentage of RAM or to the target size, whichever is greater
diff --git a/debian/patches/lp1968805-Swapon-with-maximum-priority-before-hibernation.patch b/debian/patches/lp1968805-Swapon-with-maximum-priority-before-hibernation.patch
0new file mode 100644162new file mode 100644
index 0000000..5ed11c1
--- /dev/null
+++ b/debian/patches/lp1968805-Swapon-with-maximum-priority-before-hibernation.patch
@@ -0,0 +1,63 @@
1Author: Matthew Ruffell <matthew.ruffell@canonical.com>
2Date: Wed 13 Apr 2022 15:39:15 +1200
3Description:
4
5 Swapon with maximum priority before hibernation
6
7 Since Focal, on systems with additional or multiple swapfiles
8 configured, swapon on /swap-hibinit will be added with the next
9 lowest priority for the system.
10
11 For example, if we have /swapfile configured with a default priority
12 of -2, swapon /swap-hibinit will be added with priority -3.
13
14 $ sudo swapon /swapfile
15 $ swapon --show
16 NAME TYPE SIZE USED PRIO
17 /swapfile file 4G 0B -2
18 $ sudo swapon /swap-hibinit
19 $ swapon --show
20 NAME TYPE SIZE USED PRIO
21 /swapfile file 4G 0B -2
22 /swap-hibinit file 3.9G 0B -3
23
24 This causes hibernation to fail, since hibernation targets the
25 highest priority swapfile, which is no longer /swap-hibinit, and the
26 kernel command line is specifically configured to resume from the
27 UUID and offset for /swap-hibinit.
28
29 The result is that EC2 console says "Stopping" for 20 minutes before
30 the instance is force stopped.
31
32 Fix this by setting /swap-hibinit to maximum possible priority,
33 32767, as we swapon right before hibernation takes place.
34
35Bug-Ubuntu: https://bugs.launchpad.net/bugs/1968805
36Forwarded: yes, https://github.com/aws/amazon-ec2-hibinit-agent/commit/a2303d269610a6e7415c5045766da605eaa7e30f
37Last-Update: 2022-04-13
38
39Index: ec2-hibinit-agent-1.0.0/acpid.sleep.sh
40===================================================================
41--- ec2-hibinit-agent-1.0.0.orig/acpid.sleep.sh 2022-04-13 15:55:57.642062380 +1200
42+++ ec2-hibinit-agent-1.0.0/acpid.sleep.sh 2022-04-13 15:55:57.642062380 +1200
43@@ -2,6 +2,11 @@
44
45 #PATH=/sbin:/bin:/usr/bin
46
47+# Hibernation selects the swapfile with highest priority. Since there may be
48+# other swapfiles configured, ensure /swap-hibinit is selected as hibernation
49+# target by setting to maximum priority. See LP#1968805.
50+swap_priority=32767
51+
52 do_hibernate() {
53 if [ -d /run/systemd/system ]; then
54 systemctl hibernate
55@@ -14,7 +19,7 @@
56
57 case "$2" in
58 SBTN)
59- swapon /swap-hibinit && do_hibernate
60+ swapon --priority=$swap_priority /swap-hibinit && do_hibernate
61 ;;
62 *)
63 logger "ACPI action undefined: $2" ;;
diff --git a/debian/patches/lp2023924-remove-quotes-from-state-dir.patch b/debian/patches/lp2023924-remove-quotes-from-state-dir.patch
0new file mode 10064464new file mode 100644
index 0000000..d178d21
--- /dev/null
+++ b/debian/patches/lp2023924-remove-quotes-from-state-dir.patch
@@ -0,0 +1,19 @@
1Description: Remove quotes in config file for state_dir value
2It creates strange directory /"
3Author: Anchal Agarwal <anchalag@amazon.com>
4Origin: backport, https://github.com/aws/amazon-ec2-hibinit-agent/commit/d87d6365009f1663247fd8595b487c59d9b121f1
5Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/ec2-hibinit-agent/+bug/2023924
6Last-Update: 2023-06-20
7---
8This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
9--- a/etc/hibinit-config.cfg
10+++ b/etc/hibinit-config.cfg
11@@ -12,7 +12,7 @@
12 touch-swap = False
13
14 # Location where to create any state files
15-state-dir = "/var/lib/hibinit-agent"
16+state-dir = /var/lib/hibinit-agent
17
18 [swap]
19 # If there's no swap then we create it to be equal to the specified
diff --git a/debian/patches/series b/debian/patches/series
index 136d930..1555533 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -7,3 +7,6 @@ detect-hibernate-cmd-by-default.patch
70008-Detect-disabled-hibernation-with-Python-3-s-urllib-t.patch70008-Detect-disabled-hibernation-with-Python-3-s-urllib-t.patch
80008-Always-set-resume-device-by-PARTUUID-instead-of-by-d.patch80008-Always-set-resume-device-by-PARTUUID-instead-of-by-d.patch
90010-Update-grub-configuration-when-it-needs-an-update.patch90010-Update-grub-configuration-when-it-needs-an-update.patch
10lp1968805-Swapon-with-maximum-priority-before-hibernation.patch
11lp1941785-Add-support-for-IMDSv2.patch
12lp2023924-remove-quotes-from-state-dir.patch

Subscribers

People subscribed via source and target branches