Merge lp:~brunonova/oneconf/lp1165104 into lp:oneconf

[Bruno Nova]

These commits try to fix bug LP: #1165104. I tested this in my laptop (and Virtualbox) and it seems to be working for me (in Saucy).

I tested this by running "oneconf-service --debug" right from the source code directory, without installing (and, if it was already running, I execute "oneconf-query --stop" first).
With the service running, I watch its debug output, start the Ubuntu Software Center (USC), make sure it's configured to synchronize (clicking the "File > Sync" option in the menu) and check to see if it displays the correct sync results (if it shows the other computers and if it can show a comparison of what is and is not installed in them).
If it's not working yet, I quit USC and restart the oneconf service, and try again (it may sync what's missing).
I did this in my laptop and in Virtualbox (with an Ubuntu Live DVD).

It is also useful to check the files in ~/.cache/oneconf/LONG_HEX_NAME/.

The two changes where "Ubuntu Software Center" was renamed to "Ubuntu One" are meant only for Saucy and newer releases.
The comments in the bug report have more info.

[Didier Roche-Tolomelli (didrocks)]

Thanks for this fix proposal!

Your fixes sounds legit and should work. However, I'm puzzle because running directly your branch or even building the package, I'm still getting:
ERROR:oneconf.networksync:Invalid machine list from server, stopping sync: 401: {'date': 'Fri, 13 Dec 2013 09:14:16 GMT', 'etag': '"7e287fc885d9fdcf42da3a12f38572c1"', 'content-type': 'text/html; charset=utf-8', 'last-modified': 'Fri, 13 Dec 2013 09:14:16 GMT', 'cache-control': 'max-age=0', 'server': 'Apache/2.2.22 (Ubuntu)', 'www-authenticate': 'OAuth realm="Ubuntu Software Center"', 'content-length': '22', 'status': '401', 'expires': 'Fri, 13 Dec 2013 09:14:16 GMT', 'vary': 'Authorization', 'strict-transport-security': 'max-age=2592000'}

I grepped into the oneconf repository and I don't see any trace of "Ubuntu Software Center" token. I won't have time to look at it today (holidays and a lot of touch stuff to catchup), but if you have any time to have a deeper look, that will be great!

I tried:
- latest trusty
- bzr branch <your branch>
- ./oneconf-service --debug

I tried as well:
- bzr branch <your branch>
- bzr bd
- install the .debs (checking that the python3 binary files were updated)
- /usr/share/oneconf/oneconf-service --debug

Did you change any other file on your system for the cred testing?
Once I'm back from holidays, I'll give it another look. If you have time before me, it would be great to see that fixed ;)

[Bruno Nova (brunonova)]

In the bug comments, Barry said he gets the same 401 error on "list-machines/", using Trusty as well.
I'm using Saucy now (when I started trying to fix this I was in Raring). I haven't tested in Trusty.

My first bet for that 401 error was "Is the server accepting requests from computers running Trusty yet?", BUT the response includes 'OAuth realm="Ubuntu Software Center"', so I don't know. I thought I had renamed all occurrences of the credential.

Answering you last question: I haven't changed any system files for that (I think). And last time I tested this, I booted a Live DVD in Virtualbox, copied the source code from the host PC, and tested directly. Obviously, I had to start Ubuntu Software Center and login to my account there (and then to test if the sync was working).

[Bruno Nova (brunonova)]

According to the latest comments in the bug report, and to my tests, this fix seems to be working for Saucy.
So I think this fix should be released for Saucy (after the usual update process, of course).

I haven't tested this in Trusty, so I suppose it's still not working there (thus the bug should be kept open for Trusty).

[James Tait (jamestait)]

Well now this is interesting. I seem to have this working on one machine running Saucy, but getting the same 401 error on another machine running Saucy. The 401 response also contains the OAuth Realm="Ubuntu Software Center".

It looks like the OAuth token I have in my keyring is no longer valid - if I look at https://login.ubuntu.com/+applications I don't see this device listed there any more. Unfortunately, I'm unable to get to a desktop on that machine currently, so I don't think I can re-login.

[James Tait (jamestait)]

Of course I can re-login:

$ ssh -X myotherhost
$ gdbus call --session --dest com.ubuntu.sso --object-path /com/ubuntu/sso/credentials --method com.ubuntu.sso.CredentialsManagement.clear_credentials "Ubuntu One" {}
$ ubuntuone-control-panel-qt

I now see the device in https://login.ubuntu.com/+applications and my package sync has worked.

[Bruno Nova (brunonova)]

Maybe the 401 error reported for Trusty in the bug comments is also due to an invalid OAuth token?
Maybe the upgrade from Raring to Saucy makes the token invalid (due to the rename)?

Also, since the commits apparently fix the issue for Saucy, maybe they should be merged and released for saucy-proposed (and trusty)?
There wasn't much activity recently.

[Didier Roche-Tolomelli (didrocks)]

Seems barry didn't get any time to get a look at it unfortunately, sorry about that. Do you know if your fix is working for trusty? That would be a great way to reuse it (as saucy is EOL)

[Bruno Nova (brunonova)]

I had created a package for Trusty and put it in a PPA (the comments in the bug report mention that), and I am using it.
So yes, it's working. Or at least it was. :)

I think I based the package on the latest development version.
This merge proposal is based on an older version (back in Saucy).

But hold on a moment. I'll test this now to see if it still works.

[Bruno Nova (brunonova)]

I tested it now with another computer (Trusty live session), and it seems to be working.

In that computer I pulled lp:oneconf and merged with my branch, then started oneconf-service directly from the source tree.
In my laptop, I used the package I created that was already installed.
I started/restarted oneconf-service everytime I wanted to force a sync.
The sync was successful and I could see see the results in both computers.

Though I must say that the sync/package comparison doesn't work 100% perfectly.
The comparison doesn't show all applications that differ.
For example, I installed Geany, then forced a sync in both computers, and it didn't appear in the comparison. Don't know if that has anything to do with Geany being in the (not enabled by default) universe repository.
But this is outside of the scope of this fix.

[Bruno Nova (brunonova)]

Didier Roche, you are not set as "Reviewer", but could you, or anyone else, review this and finally merge it into Utopic and Trusty if it's good?
As stated above, the fix seems to be working in Trusty.

[Iain Lane (laney)]

I've merged this after testing that I can get another machine to show up with this change (I needed a further fix to the hostname regex, but that is separate).

Thank you for the fix, and I'm really sorry that it's taken so long to review it.

I'll upload this to 'wily' once it opens. We could perform a stable update to 14.04/15.04 if you want - if so, please update the bug with the information in https://wiki.ubuntu.com/StableReleaseUpdates#Procedure - I'm subscribing to the bug so I'll see it.

review: Approve

[Bruno Nova (brunonova)]

Wow! Nice! Better late than never! :)
(I'm not mad. You guys are doing a lot of stuff at the same time!)

I'll do what you asked and update the bug description in a few days.

=== modified file 'oneconf/networksync/__init__.py'
--- oneconf/networksync/__init__.py	2012-12-13 23:37:23 +0000
+++ oneconf/networksync/__init__.py	2013-12-10 16:01:38 +0000
@@ -103,7 +103,7 @@
                 token_secret=credential['token_secret'],
                 consumer_key=credential['consumer_key'],
                 consumer_secret=credential['consumer_secret'],
-                oauth_realm='Ubuntu Software Center')
+                oauth_realm='Ubuntu One')
             self.infraclient = WebCatalogAPI(service_root=service_root,
                                              auth=authorizer)
         self._refresh_can_sync()

=== modified file 'oneconf/networksync/infraclient_pristine.py'
--- oneconf/networksync/infraclient_pristine.py	2012-11-21 20:01:42 +0000
+++ oneconf/networksync/infraclient_pristine.py	2013-12-10 16:01:38 +0000
@@ -29,6 +29,11 @@
 #    default_service_root = 'http://localhost:8000/cat/api/1.0'
     default_service_root = 'https://apps.staging.ubuntu.com/cat/api/1.0'
     default_content_type = 'application/x-www-form-urlencoded'
+    
+    def _get(self, *args, **kwargs):
+        """Return super's _get() as str, converting it from bytes if necessary."""
+        res = super()._get(*args, **kwargs)
+        return res.decode() if type(res) == bytes else res
 
     @returns_json
     def server_status(self):
@@ -83,7 +88,7 @@
             raise APIError('Package list empty')
         # FIXME: need to do this hack to transform the http request to a json format content
         try:
-            package_list = json.loads(package_list[1:-1].replace("'", '"').replace("True", "true").replace("False", 'false'))
+            package_list = ast.literal_eval(package_list[1:-1])
         except ValueError as e:
             raise APIError('Package list invalid: %s' % e)
         return package_list

=== modified file 'oneconf/networksync/ssohandler.py'
--- oneconf/networksync/ssohandler.py	2012-11-30 15:32:38 +0000
+++ oneconf/networksync/ssohandler.py	2013-12-10 16:01:38 +0000
@@ -50,7 +50,7 @@
 
         # use USC credential
         #self.appname = _("Ubuntu Software Center Store")
-        self.appname = "Ubuntu Software Center"
+        self.appname = "Ubuntu One"
 
         if "ONECONF_SSO_CRED" in os.environ:
             if os.environ["ONECONF_SSO_CRED"].lower() == 'true':