Merge ~antonioromito/cloud-init:cloud-init-BZ_1673170 into cloud-init:master

[Antonio Romito]

cmd:main.py: Fix missing 'modules-init' key in modes dict

Cloud-init's main.py will fail when presented with a new
stage name 'modules-init' if upgrading an older cloud-init.
Fix this by initializing unknown stage names before accessing.

LP: #1815109

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:2b2588e7168ef0fa2661bb99b136d3e322d56b5d
https://jenkins.ubuntu.com/server/job/cloud-init-ci/559/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/559/rebuild

review: Approve (continuous-integration)

[Antonio Romito (antonioromito)]

Found the root cause.

Better to go for the following solution:

# diff /usr/lib/python2.7/site-packages/cloudinit/cmd/main.py.orig /usr/lib/python2.7/site-packages/cloudinit/cmd/main.py
643,644c643,646
< elif mode not in status['v1']:
< status['v1'][mode] = nullstatus.copy()
---
> else:
> for m in modes:
> if m not in status['v1']:
> status['v1'][m] = nullstatus.copy()

[Dan Watkins (oddbloke)]

review: Needs Information

[Antonio Romito (antonioromito)]

Probably the description of what the issue is can be found at: https://access.redhat.com/solutions/3896071

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:e9fab8a5cd8d7537deaa23a03c229d54e459e8a3
https://jenkins.ubuntu.com/server/job/cloud-init-ci/656/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/656/rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:e8a28b7a76089aa81caedd67ff995d8a5d18ffc6
https://jenkins.ubuntu.com/server/job/cloud-init-ci/657/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/657/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

review: Approve

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Autolanding.
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/211/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    FAILED: Ubuntu LTS: Integration

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

The autolander didn't like some of the UTF-8 chars in the commit message. I've updated the commit message to summarize the finding and remove the UTF-8 chars (we'll fix the autolander as well).

This should get things going again.

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py
index 933c019..a5446da 100644
--- a/cloudinit/cmd/main.py
+++ b/cloudinit/cmd/main.py
@@ -632,13 +632,14 @@ def status_wrapper(name, args, data_d=None, link_d=None):
         'start': None,
         'finished': None,
     }
+
     if status is None:
         status = {'v1': {}}
-        for m in modes:
-            status['v1'][m] = nullstatus.copy()
         status['v1']['datasource'] = None
-    elif mode not in status['v1']:
-        status['v1'][mode] = nullstatus.copy()
+
+    for m in modes:
+        if m not in status['v1']:
+            status['v1'][m] = nullstatus.copy()
 
     v1 = status['v1']
     v1['stage'] = mode