Merge lp:~charlesk/keeper/tar-creator-boogs into lp:keeper/devel

[Charles Kerr]

Experimental branch, not for commit

I'm not getting enough traction on the lp:~charlesk/keeper/tar-creator-edge-cases branch, so this branch is a testing ground to add pieces from that branch to see what change is making the CI bot so unhappy.

[unity-api-1-bot (unity-api-1-bot)]

FAILED: Continuous integration, rev:109
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/75/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/661/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/667
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/482/console
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/482/console

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/75/rebuild

review: Needs Fixing (continuous-integration)

[lp:~charlesk/keeper/tar-creator-boogs]

110. By Charles Kerr on 2016-09-15

re-enable libarchive's internal buffering

[unity-api-1-bot (unity-api-1-bot)]

PASSED: Continuous integration, rev:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/79/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/667
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/673
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/488/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/488
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/488/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/79/rebuild

review: Approve (continuous-integration)

[unity-api-1-bot (unity-api-1-bot)]

FAILED: Continuous integration, rev:110
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/80/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build/668/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/674
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/489/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/489/artifact/output/*zip*/output.zip
    FAILURE: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/489/console
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/489/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/489/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/489/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/489/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/489/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/489
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/489/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-keeper-ci/80/rebuild

review: Needs Fixing (continuous-integration)

Unmerged revisions

110. By Charles Kerr on 2016-09-15

re-enable libarchive's internal buffering

109. By Charles Kerr on 2016-09-15

use new helper wrapped_archive_write_new() so that all our archive objects are created in a consistent way

=== modified file 'src/tar/tar-creator.cpp'
--- src/tar/tar-creator.cpp	2016-09-05 18:38:36 +0000
+++ src/tar/tar-creator.cpp	2016-09-15 17:26:09 +0000
@@ -68,10 +68,8 @@
         // if this is the first step, create an archive
         if (!step_archive_)
         {
-            step_archive_.reset(archive_write_new(), [](struct archive* a){archive_write_free(a);});
-            archive_write_set_format_pax(step_archive_.get());
-            if (compress_)
-                archive_write_add_filter_xz(step_archive_.get());
+qDebug() << "new archive";
+            step_archive_ = wrapped_archive_write_new(compress_);
             archive_write_open(step_archive_.get(), &step_buf_, nullptr, append_bytes_write_cb, nullptr);
 
             step_file_.reset();
@@ -202,20 +200,18 @@
     {
         ssize_t archive_size {};
 
-        auto a = archive_write_new();
-        archive_write_set_format_pax(a);
-        archive_write_open(a, &archive_size, nullptr, count_bytes_write_cb, nullptr);
+        auto a = wrapped_archive_write_new(false);
+        archive_write_open(a.get(), &archive_size, nullptr, count_bytes_write_cb, nullptr);
 
         for (const auto& filename : filenames_)
         {
-            add_file_header_to_archive(a, filename);
+            add_file_header_to_archive(a.get(), filename);
 
             // libarchive pads any missing data,
             // so we don't need to call archive_write_data()
         }
 
-        archive_write_close(a);
-        archive_write_free(a);
+        archive_write_close(a.get());
         return archive_size;
     }
 
@@ -223,14 +219,12 @@
     {
         ssize_t archive_size {};
 
-        auto a = archive_write_new();
-        archive_write_set_format_pax(a);
-        archive_write_add_filter_xz(a);
-        archive_write_open(a, &archive_size, nullptr, count_bytes_write_cb, nullptr);
+        auto a = wrapped_archive_write_new(true);
+        archive_write_open(a.get(), &archive_size, nullptr, count_bytes_write_cb, nullptr);
 
         for (const auto& filename : filenames_)
         {
-            add_file_header_to_archive(a, filename);
+            add_file_header_to_archive(a.get(), filename);
 
             // process the file
             QFile file(filename);
@@ -242,7 +236,7 @@
                 if (n_read == 0)
                     break;
                 if (n_read > 0)
-                    archive_write_data(a, buf, size_t(n_read));
+                    archive_write_data(a.get(), buf, size_t(n_read));
                 if (n_read < 0) {
                     auto errstr = QStringLiteral("Reading '%1' returned %2 (%3)")
                                       .arg(file.fileName())
@@ -254,11 +248,20 @@
             }
         }
 
-        archive_write_close(a);
-        archive_write_free(a);
+        archive_write_close(a.get());
         return archive_size;
     }
 
+    static std::shared_ptr<struct archive> wrapped_archive_write_new(bool compress)
+    {
+        auto archive = archive_write_new();
+        archive_write_set_format_pax(archive);
+        //archive_write_set_bytes_per_block(archive, 0);
+        if (compress)
+            archive_write_add_filter_xz(archive);
+        return std::shared_ptr<struct archive>(archive, [](struct archive* a){archive_write_free(a);});
+    }
+
     const QStringList filenames_;
     const bool compress_ {};