New MEGA.nz backend based on official "MEGAcmd" tools (patch attached)

Were the changes based on the 0.7 series or the 0.8 series?

0.7 is EOL due to Python 2 going EOL. 0.8 is the current development series.

Hi, attached "patch" has been created on top of Ubuntu 20.04 LTS "duplicity" 0.8.11.1612-1 package, and running Python 3.8.2.

But it has also been tested and is working in the same way on Ubuntu 19.10 x86_64 with Python 3.7.5, on top of Ubuntu's "duplicity 0.8.04-2ubuntu1" package.

Further to this, used the same backend on a Raspbian GNU/Linux 10 armhf with Python 2.7.16 , on top of Raspbian's "duplicity 0.7.18.2-1" package, and other than removing the reference to "subprocess.check_output / timeout" and "subprocess.TimeoutExpired" (which don't exist in Python earlier than 3.3) the backend also works.

Finally, since the time I submitted this Bug, I made a minor change to the code, to discard "mega-login" progress output, which looks as simple as this:
"""
--- megav2backend.py 2020-05-09 11:05:52.382811192 +0200
+++ megav2backend_v1.1.py 2020-05-17 19:36:23.891145593 +0200
@@ -150,7 +150,7 @@
         except Exception as e:
             cmd = ['mega-login', self._username, self._password]
             try:
- subprocess.check_output(cmd)
+ subprocess.check_output(cmd, stderr=subprocess.DEVNULL)
             except Exception as e:
                 raise BackendException("Could not log in to MEGA, error : '%s'" % (e,))
"""

Other than that, the only "restriction" I have found in the submitted backend is, to workaround "mega-session" sometimes not returning at all, the "mega_login" function does a "subprocess.check_output" with a hardcoded 30 second timeout (timeout=30), which is only available for Python 3.3 and later as far as I know. But I guess at this time in history, Python 3.3 is not precisely bleeding edge and hence will not result in a problem for people resorting to this backend for storing duplicity backups. Can't remember if "stderr=subprocess.DEVNULL" is Python 3.3+ or Python 3.5+ though.

The code has moved on a lot since those versions. In fact, the removal of the unicode adornment on all the strings is problematic. It causes all sorts of problems with diffing the files and trying to isolate your changes. I've attached a sample diff to show point.

As to Python 3.3, duplicity has only been tested to work on Python 3.6 and above. All of the ones before that have oddball unicode issues.

The current source resides at https://gitlab.com/duplicity/duplicity. Given the number of incompatible changes, I'm going to need some help. If you could checkout that code and apply your changes, I would really appreciate it, otherwise, it will take a long while.

Diff between commandline.py versions.

No problem, give me some time to adapt the backend based on the latest code as in Gitlab and will post here once I have it tested on my computers for a few days.

I think I have it. It took more time to workaround several issues with my Ubuntu install which were preventing the Debian package from being created than doing the changes itself and testing.

Testing was done with very short-lived backups (to avoid having to wait for days), but it should make no difference. Testing proved two problems (bugs) with the new backend, which I have now hopefully fixed:
- Missing "decode()" for "_delete", and the usual mismatch between b'' and u''
- Had to use "-f" for "mega-rm", or else deleted files end up in the MEGA account Trash, which is obviously not correct for a backup backend

Diff attached was created against:
"""
commit ff4fb49512e3593156f348bc1306f308303d69e0 (origin/master, origin/HEAD)
Author: Kenneth Loafman <email address hidden>
Date: Wed May 20 15:22:37 2020 -0500
"""

Please have a look into it and tell me if more work on this is necessary. Developed and tested on Ubuntu 20.04 LTS running x86_64 and Python 3.8.2.

Looks good! I just committed to GitLab after making some minor code style changes.

Thanks much for the hard work!

To you for the software and the time to make the code adhere to proper coding style.

Doing further testing and using MEGA free tier run out of space and hence I tought it would be good to properly handle these situations from the code raising a backend exception when occurring.

So I caught the error on "upload()"ing any file and returned error and raised backend exception, printing the corresponding error message.

Did for both the old MEGA and the new MEGAv2 backends and tested. Seems to work fine.

Please apply attached patch against (hopefully) current head.

Thank you.