Duplicity

Dropbox backend: Upload last part using files_upload_session_finish

Bug #1573957 reported by Henrik Farre on 2016-04-23

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Duplicity	Fix Released	Medium	Unassigned	Duplicity 0.7.08

Bug Description

Duplicity version: 0.7.07-1
Python version: 2.7.11-3
OS Distro and version: Arch Linux
Type of target filesystem: Dropbox

In duplicity/backends/dpbxbackend.py self.api_client.files_upload_session_finish is always called with 0 data.

I have attached a patch that uploads the last chunk using self.api_client.files_upload_session_finish so calls to dropbox are reduced with one :)

I'm not a python developer so bear with me :)

Revision history for this message

Henrik Farre (henrik-rockhopper) wrote on 2016-04-23:

dpbxbackend.patch Edit (1.3 KiB, text/plain)

Revision history for this message

Dmitry Nezhevenko (dmitry.nezhevenko) wrote on 2016-05-30:

01_dpbx_fix_chunked_upload.patch Edit (2.0 KiB, text/plain)

Hi,

Initially I've coded backend with zero-size flush just to make sure everything is ok (because I was getting corrupted files with old code).

Your patch relies on assumption that read(2) always returns requested amount of bytes (except EOF) which is not true.

I'm attaching two patches:
01. uses files_upload_session_finish to avoid extra request for large files
02. uses non-chunked files_upload call for small files (less than chunk size).

passes RUN_CODE_TESTS=1 and backendtest.

Revision history for this message

Dmitry Nezhevenko (dmitry.nezhevenko) wrote on 2016-05-30:

02_dpbx_use_nonchunked_api_for_small_files.patch Edit (3.0 KiB, text/plain)

use non-chunked upload for small files

Revision history for this message

Dmitry Nezhevenko (dmitry.nezhevenko) wrote on 2016-05-30:

02_dpbx_use_nonchunked_api_for_small_files_v2.patch Edit (2.9 KiB, text/plain)

second patch, really passes RUN_CODE_TESTS=1.

So I propose to commit followed two patches:
01_dpbx_fix_chunked_upload.patch
02_dpbx_use_nonchunked_api_for_small_files_v2.patch

Kenneth Loafman (kenneth-loafman) on 2016-05-30

Changed in duplicity:
importance:	Undecided → Medium
milestone:	none → 0.7.08
status:	New → Fix Committed

Kenneth Loafman (kenneth-loafman) on 2016-07-02

Changed in duplicity:
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Patches

Add patch

Remote bug watches

Bug watches keep track of this bug in other bug trackers.