Merge ~tore.lonoy/cloud-init:hashed-chpasswd-support into cloud-init:master

[Tore]

Add support for setting hashed passwords

This change will add support for hashed passwords in cc_set_passwords.
It checks if a password is a hash with this if/regxp statement:
re.match(r'\$[1,2a,2y,5,6](\$.+){2}', p) is not None and ":" not in p:

chpasswd needs to know if the password is hashed or not, so two lists
is created so chpasswd is feed with the correct one.

Currently only static or randomly generated passwords are supported,
adding support for hashed passwords enables the user to store password
in they configuration a little more secure then using plain-text.

I have been unable to run all the tests on the change, but I've verified
that it works by running these test manually:
* setting static password
* setting static and random
* setting hashed password
* setting hashed and static password

This is my first commit to any open-source project.

[Scott Moser (smoser)]

this looks good. I marked 'Needs Fixing', just for a few nit picks.

I'd like some sort of unit test too.

The trend for adding unit tests is really to make 'handle' do less and move things into methods and use those methods.

review: Needs Fixing

6807679... by Tore on 2016-11-08

Changed regexp from regular match to compile -> match.

Looping through a large number of username/pass entries
in chpasswd would be less efficient without this change.

fde5f5c... by Tore on 2016-11-08

Improved the documentation for hashed-password support in chpasswd

[Tore (tore.lonoy)]

I've made the changes you requested.

I'm not able to create a unit test, as I really don't know how to do it. I don't see any test written already for setting password for chpasswd, so I'm little out of the blue on where to start.

[Scott Moser (smoser)]

Tore,

Thanks again, and sorry for the delay in getting back here.

In order to take this, you'll need to sign the contributors agreement (https://www.ubuntu.com/legal/contributors). See the hacking section on readthedocs for more info
http://cloudinit.readthedocs.io/en/latest/topics/hacking.html .

Also, please feel free to ping me in Freenode #cloud-init or /query me, or via email.

Taking a quick look, it looks like cc_spacewalk.py and its tests in unittests/test_handler/test_handler_spacewalk.py might be a reasonable template to start with.

Please do feel free to reach out, this looks great.
Scott

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:fde5f5c47888a0bd713bd44c19e2eb86ed5749dc
https://jenkins.ubuntu.com/server/job/cloud-init-ci/85/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-arm64/85/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-ppc64el/85/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=metal-s390x/85/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-amd64/85/console
    FAILURE: https://jenkins.ubuntu.com/server/job/cloud-init-ci/nodes=vm-i386/85/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/85/rebuild

review: Needs Fixing (continuous-integration)

[Scott Moser (smoser)]

Tore,
I went ahead and modified some of the integration tests to have a test for this.
and then proposed https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/320986 for merge into your branch.

Can you review that ?

Also, i did nto realize before that you hadn't signed the contributors agreement.
Please do so as described in HACKING.rst

[Tore (tore.lonoy)]

> Tore,
> I went ahead and modified some of the integration tests to have a test for
> this.
> and then proposed https://code.launchpad.net/~smoser/cloud-init/+git/cloud-
> init/+merge/320986 for merge into your branch.
>
> Can you review that ?

Thanks the tests, didn't realise it what that easy

>
> Also, i did nto realize before that you hadn't signed the contributors
> agreement.
> Please do so as described in HACKING.rst

I've just done so, but I believe I already have signed this. I assume this [0] is the link that should be used ?

[0] https://www.ubuntu.com/legal/contributors/submit

[Scott Moser (smoser)]

Ok thanks!

On March 25, 2017 1:20:58 AM EDT, Tore <email address hidden> wrote:
>> Tore,
>> I went ahead and modified some of the integration tests to have a
>test for
>> this.
>> and then proposed
>https://code.launchpad.net/~smoser/cloud-init/+git/cloud-
>> init/+merge/320986 for merge into your branch.
>>
>> Can you review that ?
>
>Thanks the tests, didn't realise it what that easy
>
>>
>> Also, i did nto realize before that you hadn't signed the
>contributors
>> agreement.
>> Please do so as described in HACKING.rst
>
>I've just done so, but I believe I already have signed this. I assume
>this [0] is the link that should be used ?
>
>[0] https://www.ubuntu.com/legal/contributors/submit
>--
>https://code.launchpad.net/~tore.lonoy/cloud-init/+git/cloud-init/+merge/310051
>You are reviewing the proposed merge of
>~tore.lonoy/cloud-init:hashed-chpasswd-support into cloud-init:master.

diff --git a/cloudinit/config/cc_set_passwords.py b/cloudinit/config/cc_set_passwords.py
index 6fc0051..5dd4b01 100755
--- a/cloudinit/config/cc_set_passwords.py
+++ b/cloudinit/config/cc_set_passwords.py
@@ -35,7 +35,8 @@ If the ``list`` key is provided, a list of
 ``username:password`` pairs can be specified. The usernames specified
 must already exist on the system, or have been created using the
 ``cc_users_groups`` module. A password can be randomly generated using
-``username:RANDOM`` or ``username:R``. Password ssh authentication can be
+``username:RANDOM`` or ``username:R``. A hashed password can be specified
+using ``username:$6$salt$hash``. Password ssh authentication can be
 enabled, disabled, or left to system defaults using ``ssh_pwauth``.
 
 .. note::
@@ -59,11 +60,13 @@ enabled, disabled, or left to system defaults using ``ssh_pwauth``.
     chpasswd:
         list:
             - user1:password1
-            - user2:Random
+            - user2:RANDOM
             - user3:password3
             - user4:R
+            - user4:$6$rL..$ej...
 """
 
+import re
 import sys
 
 from cloudinit.distros import ug_util
@@ -105,24 +108,46 @@ def handle(_name, cfg, cloud, log, args):
     errors = []
     if plist:
         plist_in = []
+        hashed_plist_in = []
+        hashed_users = []
         randlist = []
         users = []
+
+        prog = re.compile(r'\$[1,2a,2y,5,6](\$.+){2}')
+
         for line in plist.splitlines():
             u, p = line.split(':', 1)
             if p == "R" or p == "RANDOM":
                 p = rand_user_password()
                 randlist.append("%s:%s" % (u, p))
-            plist_in.append("%s:%s" % (u, p))
-            users.append(u)
+            elif prog.match(p) is not None and ":" not in p:
+                hashed_plist_in.append("%s:%s" % (u, p))
+                hashed_users.append(u)
+            else:
+                plist_in.append("%s:%s" % (u, p))
+                users.append(u)
 
         ch_in = '\n'.join(plist_in) + '\n'
-        try:
-            log.debug("Changing password for %s:", users)
-            util.subp(['chpasswd'], ch_in)
-        except Exception as e:
-            errors.append(e)
-            util.logexc(log, "Failed to set passwords with chpasswd for %s",
-                        users)
+        if users:
+            try:
+                log.debug("Changing password for %s:", users)
+                util.subp(['chpasswd'], ch_in)
+            except Exception as e:
+                errors.append(e)
+                util.logexc(log,
+                            "Failed to set passwords with chpasswd for %s",
+                            users)
+
+        hashed_ch_in = '\n'.join(hashed_plist_in) + '\n'
+        if hashed_users:
+            try:
+                log.debug("Setting hashed password for %s:", hashed_users)
+                util.subp(['chpasswd', '-e'], hashed_ch_in)
+            except Exception as e:
+                errors.append(e)
+                util.logexc(log,
+                            "Failed to set hashed passwords with chpasswd for %s",
+                            hashed_users)
 
         if len(randlist):
             blurb = ("Set the following 'random' passwords\n",
diff --git a/doc/examples/cloud-config.txt b/doc/examples/cloud-config.txt
index 190029e..e6ff083 100644
--- a/doc/examples/cloud-config.txt
+++ b/doc/examples/cloud-config.txt
@@ -426,14 +426,21 @@ syslog_fix_perms: syslog:root
 #
 # there is also an option to set multiple users passwords, using 'chpasswd'
 # That looks like the following, with 'expire' set to 'True' by default.
-# to not expire users passwords, set 'expire' to 'False':
+# to not expire users passwords, set 'expire' to 'False'. Also possible
+# to set hashed password, here account 'user3' has a password it set to
+# 'cloud-init', hashed with SHA-256:
 # chpasswd:
 #  list: |
 #    user1:password1
 #    user2:RANDOM
+#    user3:$5$eriogqzq$Dg7PxHsKGzziuEGkZgkLvacjuEFeljJ.rLf.hZqKQLA
 #  expire: True
 # ssh_pwauth: [ True, False, "" or "unchanged" ]
 #
+# Hashed passwords can be generated in multiple ways, example with python3:
+# python3 -c 'import crypt,getpass; print(crypt.crypt(getpass.getpass(), crypt.mksalt(crypt.METHOD_SHA512)))'
+# Newer versions of 'mkpasswd' will also work: mkpasswd -m sha-512 password
+#
 # So, a simple working example to allow login via ssh, and not expire
 # for the default user would look like:
 password: passw0rd