Merge ~holmanb/curtin:master into curtin:master

[Brett Holman]

support older versions of iproute2

Bionic's iproute2 package contains a half-baked -json implementation; ip address and ip link work, while ip route simply accepts the flag but gives non-json output. This proposal works around that.

[Brett Holman (holmanb)]

More details in the original github PR comment.

https://github.com/canonical/curtin/pull/5

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:4b95f1b290fa5ba4fc95bb4e02ccba560df8cc5f
https://jenkins.canonical.com/server-team/job/curtin-ci/51/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/51/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/51/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/51/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/51/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/51//rebuild

review: Approve (continuous-integration)

88bce1e... by Brett Holman <email address hidden> on 2022-11-16

use older ss syntax

[Brett Holman (holmanb)]

More failures here: https://jenkins.canonical.com/server-team/view/cloud-init/job/curtin-cloud-init-sru/7/console

The '=' sign in the ss command works fine locally on Jammy, but not on Bionic. Removing that works on both.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:88bce1e4c72eaba96d0a7502aacfbc97e21e4dad
https://jenkins.canonical.com/server-team/job/curtin-ci/52/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/52/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/52/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/52/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/52/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/52//rebuild

review: Approve (continuous-integration)

c556aeb... by Brett Holman <email address hidden> on 2022-11-16

work around old iproute2 in tests

122d28d... by Brett Holman <email address hidden> on 2022-11-16

match lookup behavior of -json

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c556aebe18a63e774b0b5dd7bdc82cc55e065581
https://jenkins.canonical.com/server-team/job/curtin-ci/53/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/53/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/53/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/53/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/53/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/53//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:122d28d686bb34feec3b60358f6c5fd078c38e85
https://jenkins.canonical.com/server-team/job/curtin-ci/54/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-amd64/54/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-arm64/54/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-ppc64el/54/
    SUCCESS: https://jenkins.canonical.com/server-team/job/curtin-ci/nodes=metal-s390x/54/

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/curtin-ci/54//rebuild

review: Approve (continuous-integration)

[Brett Holman (holmanb)]

Our curtin job in Jenkins is currently[1] running against the latest iteration of this branch to validate cloud-init's 22.4 SRU. If this passes on torkoal, it should help with validating this MR.

[1] https://jenkins.canonical.com/server-team/view/cloud-init/job/curtin-cloud-init-sru/8/console

[Brett Holman (holmanb)]

It passed

diff --git a/tests/vmtests/__init__.py b/tests/vmtests/__init__.py
index 467c8a2..09c4c21 100644
--- a/tests/vmtests/__init__.py
+++ b/tests/vmtests/__init__.py
@@ -2412,17 +2412,22 @@ def boot_log_wrap(name, func, cmd, console_log, timeout, purpose):
 def get_lan_ip():
     (out, _) = util.subp(['ip', 'a'], capture=True)
     info = ip_a_to_dict(out)
-    (routes, _) = util.subp(['ip', '-json', 'route', 'show'], capture=True)
-    gwdevs = []
-    for route in json.loads(routes):
-        if route['dst'] == 'default':
-            gwdevs.append(route['dev'])
-    if len(gwdevs) > 0:
-        dev = gwdevs.pop()
-        addr = info[dev].get('inet4')[0].get('address')
-    else:
-        raise OSError('could not get local ip address')
-    return addr
+    try:
+        (routes, _) = util.subp(['ip', '-json', 'route', 'show'], capture=True)
+        gwdevs = []
+        for route in json.loads(routes):
+            if route['dst'] == 'default':
+                gwdevs.append(route['dev'])
+        if len(gwdevs) > 0:
+            dev = gwdevs.pop()
+            return info[dev].get('inet4')[0].get('address')
+    except json.decoder.JSONDecodeError:
+        # default via 10.247.8.1 dev eth0 onlink
+        for line in routes.split('\n'):
+            if "default" in line:
+                dev = line.split()[4]
+                return info[dev].get('inet4')[0].get('address')
+    raise OSError('could not get local ip address')
 
 
 def kernel_boot_cmdline_for_mac(mac):
diff --git a/tools/find-tgt b/tools/find-tgt
index e269beb..ec784f6 100755
--- a/tools/find-tgt
+++ b/tools/find-tgt
@@ -35,6 +35,8 @@ for r in json.load(sys.stdin):
   print(r["dev"])
   break
 ')
+    # on bionic, ip -json route show doesn't produce json
+    [ -n "$dev" ] || dev=$(ip route show | grep ^default | awk '{print $5}')
     [ -n "$dev" ] || { error "failed to find ipv4 device"; return 1; }
     addr=$(ip addr show dev "$dev" |
         awk '$1 == "inet" {gsub(/\/.*/,"", $2); print $2; exit}')
@@ -52,8 +54,8 @@ wait_for_tgtd() {
     local out="" pidstgt="" name="tgtd"
     while :; do
         # did we succesfully attach to the correct port?
-        out=$(ss --no-header --processes --listening --numeric "src = inet:$portal" 2>/dev/null) ||
-            fail "failed to ss --no-header --processes --listening --numeric src = inet:$portal"
+        out=$(ss --no-header --processes --listening --numeric "src inet:$portal" 2>/dev/null) ||
+            fail "failed to ss --no-header --processes --listening --numeric src inet:$portal"
         pidstgt=$(echo "$out" | awk '{ print $7; exit(0); }')
         if echo $pidstgt | grep --quiet "pid=$pid,"; then
             error "expected pid $pid is listening on $portal on try $try."