Merge lp:~milo/linaro-patchmetrics/user-migration into lp:linaro-patchmetrics

[Milo Casagrande]

A simple script to migrate users in the DB to the new Crowd authentication mechanism.
Users will be matched based on their email address.

There is a workaround at the moment in the code for emails longer than 30 chars: Django does not support that.

[Paul Sokolovsky (pfalcon)]

One thing is that I thought this would be belong to django custom management commands, but I see that patchmetrics has that patterns of bin/ utils using _pythonpath module ok.

But the strategy looks pretty surprising. I always thought of something like: when we have both old (LP) and new (Crowd) user (so user would need to login via Crowd), and we migrate all objects from one user id to another. so, we retain both users in there (as backout strategy for example).

So, approach of this script with "destructive" migration of user's identity from Launchpad to Crowd feels almost like cheating. Well, if it works, I cannot really say something bad about it, because it's indeed nice automated en-masse solution. One thing I'd mention though is matter of how complete this "identity switching" is. For example, there's likely openid data linked to this account. I don't think this would interfere with something, but well, worth mentioning.

Otherwise, look good.

review: Approve

[Milo Casagrande (milo)]

On Wed, Oct 23, 2013 at 4:36 PM, Paul Sokolovsky
<email address hidden> wrote:
> Review: Approve
>
> One thing is that I thought this would be belong to django custom management commands, but I see that patchmetrics has that patterns of bin/ utils using _pythonpath module ok.

I personally do not like it either, but I do not like either the code
base to be honest.

> But the strategy looks pretty surprising. I always thought of something like: when we have both old (LP) and new (Crowd) user (so user would need to login via Crowd), and we migrate all objects from one user id to another. so, we retain both users in there (as backout strategy for example).
>
> So, approach of this script with "destructive" migration of user's identity from Launchpad to Crowd feels almost like cheating.

You are right, it is indeed like cheating.

But the more I looked at the DB though, the more I thought that this
was the most likely and easiest solution to migrate in just one batch.
If we ended up adding new users, as I was trying to do in the
beginning, we would have had to mass-update all the teams in
patchemtrics, and the user-teams membership (and still I was not sure
if we could have guaranteed the correct results).

> Well, if it works, I cannot really say something bad about it, because it's indeed nice automated en-masse solution. One thing I'd mention though is matter of how complete this "identity switching" is. For example, there's likely openid data linked to this account. I don't think this would interfere with something, but well, worth mentioning.

I tried to clean up also those, we will not need them anymore, but it
will take a little bit of cleaning because there are external keys
involved. We can do that, probably not as programmatically as this MP,
but it can be done with some SQL queries.

As far as we are concerned, the openid data does not pose a problem
for patchmetrics to work.
For how complete it is, you should be able to get everything you had
with the old authentication method.

--
Milo Casagrande | Automation Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

[Paul Sokolovsky (pfalcon)]

> As far as we are concerned, the openid data does not pose a problem for patchmetrics to work.

I agree, so should be good to go. Yet, we should keep this in contingency plan - if something pops up, we'll need to react quickly.

=== added file 'apps/patchmetrics/bin/migrate-linaro-users.py'
--- apps/patchmetrics/bin/migrate-linaro-users.py	1970-01-01 00:00:00 +0000
+++ apps/patchmetrics/bin/migrate-linaro-users.py	2013-10-23 13:03:07 +0000
@@ -0,0 +1,70 @@
+#!/usr/bin/python
+# Copyright (C) 2013 Linaro
+#
+# Author: Milo Casagrande <[e-mail]>
+# This file is part of the Patchmetrics package.
+#
+# Patchmetrics is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Patchmetrics is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Patchmetrics; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+import _pythonpath
+import sys
+
+from django.conf import settings
+from django.contrib.auth.models import User
+
+from patchmetrics.crowd import (
+    Crowd,
+    CrowdException,
+    CrowdNotFoundException,
+)
+
+if __name__ == '__main__':
+    if settings.AUTH_CROWD_APPLICATION_USER:
+        crwd = Crowd(settings.AUTH_CROWD_APPLICATION_USER,
+                     settings.AUTH_CROWD_APPLICATION_PASSWORD,
+                     settings.AUTH_CROWD_SERVER_REST_URI)
+
+        users = User.objects.all().distinct()
+        for user in users:
+            if user and (user.is_active and user.email):
+                crowd_usr = None
+                try:
+                    crowd_usr = crwd.get_user(user.email)
+                    if crowd_usr.name != user.username:
+                        # Corner case, at least for now: Django has a limit
+                        # on the chars for username.
+                        if len(user.email) > 30:
+                            sys.stderr.write(
+                                "Impossible to save user, email is longer "
+                                "than 30 chars: {0}\n".format(user.email))
+                            continue
+                        else:
+                            split_name = crowd_usr.display_name.split()
+                            user.first_name = split_name[0]
+                            user.last_name = " ".join(split_name[1:])
+                            # Use the email from crowd, since some users looks
+                            # like have an email with capital letters that
+                            # shouldn't be.
+                            user.username = crowd_usr.emails[0]
+                            user.save()
+                except CrowdNotFoundException:
+                    # Silently ignore, user is not in Crowd we can't really do
+                    # much here.
+                    pass
+                except CrowdException, ex:
+                    sys.stderr.write("{0}\n".format(str(ex)))
+    else:
+        sys.stderr.write("No Crowd credentials found.\n")
+        sys.exit(1)