Merge ~ahasenack/ubuntu/+source/pmdk:eoan-pmdk-merge-1.6 into ubuntu/+source/pmdk:debian/experimental

[Andreas Hasenack]

PPA: https://launchpad.net/~ahasenack/+archive/ubuntu/pmdk-1.6-merge
sudo add-apt-repository ppa:ahasenack/pmdk-1.6-merge

First merge using the git ubuntu workflow after the resync done in 1.5.1 with debian. Nice and easy.
As expected, dropped a bit of delta (removing a test that no longer fails).

About the rest:
- d/p/manpage-debug-packages.patch: was accepted by debian and will be in the next package upload
- d/rules, d/not-installed, dh_missing: with --list, was accepted by debian and will be in the next package upload. We switched to --fail. I'll wait for the next release to submit that to debian.

Lintian checks:
- symbols-file-missing-build-depends-package-field: accepted by debian, will be in their next upload
- spelling-error-in-manpage: accepted by upstream, will be in the next upstream release

DEP8 tests, simple as they are, pass locally.

Debian didn't document their changes properly in d/changelog for their 1.6 upload, but a git diff between sid and experimental shows:
- dropping the removal of a test
- adding of strace to build-depends. I believe this is for a unittest. It doesn't become a runtime dependency. Maybe related to the valgrind test we are dropping? I'm not sure. I didn't find many clues in upstream's git repo.

In parallel, I'm also making an MP for libpmemobj-cpp to bump it to 1.6, but there doesn't seem to be a hard dep between the two having to be in sync.

[Andreas Hasenack (ahasenack)]

Let me check if the tests pass *with* valgrind in this version, then we can drop that bit of delta.

[Andreas Hasenack (ahasenack)]

Nope, valgrind still fails: https://launchpadlibrarian.net/424868941/buildlog_ubuntu-eoan-amd64.pmdk_1.6-1ubuntu1~ppa2_BUILDING.txt.gz

Let's keep it dropped from d/control.

[Christian Ehrhardt  (paelzer)]

It seems upstreaming on this worked great so far and upstream as well as Debian are responsive.
Be sure to mention that when (re)opening the MIR later.

The one thing I'd like to mention/improve (as we have done on libpmemobj-cpp) is that we could/should mention that the transitions can be dropped after 20.04. That can be done this merge or anytime later - up to you.

I'm not concerned on the strace change as long as it is only build-dep.

I wondered about future remaining Delta, it seems to be
- README.source which I think would apply to Debian as well nowadays. Maybe propose that
  to Debian together with the --fail-missing later this cycle.
- valgrind; well lets not do too mcuh for now, just give it a try every merge and keep it if
  still failing
- d/NEWS: this does not apply to Debian but is shown when "passing" this version on an upgrade.
  Hence it is part of the same "can be dropped after 20.04" that we could mention on the commit.

That is one potential typo and a lot of thought for the future - but +1 for the current MP.

review: Approve

119fcdf... by Andreas Hasenack on 2019-05-23

merge-changelogs

74fdf2d... by Andreas Hasenack on 2019-05-23

reconstruct-changelog

a7b5309... by Andreas Hasenack on 2019-05-23

update-maintainer

[Andreas Hasenack (ahasenack)]

Tagged and uploaded.
$ git push pkg upload/1.6-1ubuntu1
Enumerating objects: 57, done.
Counting objects: 100% (57/57), done.
Delta compression using up to 4 threads
Compressing objects: 100% (50/50), done.
Writing objects: 100% (50/50), 8.41 KiB | 1.68 MiB/s, done.
Total 50 (delta 29), reused 4 (delta 0)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/pmdk
 * [new tag] upload/1.6-1ubuntu1 -> upload/1.6-1ubuntu1

$ dput ubuntu ../pmdk_1.6-1ubuntu1_source.changes
Checking signature on .changes
gpg: ../pmdk_1.6-1ubuntu1_source.changes: Valid signature from AC983EB5BF6BCBA9
Checking signature on .dsc
gpg: ../pmdk_1.6-1ubuntu1.dsc: Valid signature from AC983EB5BF6BCBA9
Package includes an .orig.tar.gz file although the debian revision suggests
that it might not be required. Multiple uploads of the .orig.tar.gz may be
rejected by the upload queue management software.
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading pmdk_1.6-1ubuntu1.dsc: done.
  Uploading pmdk_1.6.orig.tar.gz: done.
  Uploading pmdk_1.6-1ubuntu1.debian.tar.xz: done.
  Uploading pmdk_1.6-1ubuntu1_source.buildinfo: done.
  Uploading pmdk_1.6-1ubuntu1_source.changes: done.
Successfully uploaded packages.

[Andreas Hasenack (ahasenack)]

This migrated.

diff --git a/debian/NEWS b/debian/NEWS
new file mode 100644
index 0000000..22e63d1
--- /dev/null
+++ b/debian/NEWS
@@ -0,0 +1,19 @@
+pmdk (1.5.1-1ubuntu1) eoan; urgency=medium
+
+  Important changes introduced in version 1.5. See
+  http://pmem.io/2018/10/22/release-1-5.html for details.
+
+  The libpmemcto code has been removed from upstream in this release, and as
+  such the corresponding libpmemcto* Debian packages are no longer being
+  built.
+
+  The c++ header files and documentation for libpmemobj were moved to a new
+  source package called libpmemobj-cpp, and are present in the binary packages
+  libpmemobj-cpp-dev and libpmemobj-cpp-doc respectively.
+
+  As a side effect of performance optimizations, the libpmemobj on-media layout
+  had to be changed, which means that old pools have to be converted using
+  pmdk-convert.
+
+
+ -- Andreas Hasenack <[e-mail]>  Fri, 10 May 2019 18:38:26 +0000
diff --git a/debian/README.source b/debian/README.source
new file mode 100644
index 0000000..6574150
--- /dev/null
+++ b/debian/README.source
@@ -0,0 +1,33 @@
+lintian check: debian-watch-does-not-check-gpg-signature
+========================================================
+Upstream does not distribute detatched gpg signatures
+
+https://github.com/pmem/issues/issues/848
+
+
+lintian check: hardening-no-fortify-functions
+=============================================
+This check is raising warnings for the -debug library packages. As their
+respective description says, these packages are meant for special debugging
+cases and should not be installed or used outside of a debugging session.
+Therefore, this lintian warning is not being addressed for the libraries in the
+-debug packages.
+
+Also note that just installing the -debug packages is not enough to have
+applications use them, since they are installed in a path outside of the linker
+search path. The devel manpages for each library explain how to use them.
+
+
+Bundled jemalloc
+================
+The upstream source includes jemalloc version 3.6.0, released 2014-03-31, with
+local modifications, as declared in src/jemalloc/README.libvmem.
+
+Questioned if this was still necessary, and why the changes couldn't be sent
+back to jemalloc, upstream had this to say
+(https://bugs.launchpad.net/ubuntu/+bug/1752378/comments/72):
+
+"""
+We can't upstream those changes, because jemalloc changed too much since we
+forked it.
+"""
diff --git a/debian/changelog b/debian/changelog
index a756b8f..1a06551 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,29 @@
+pmdk (1.6-1ubuntu1) eoan; urgency=medium
+
+  * Merge with Debian unstable. Remaining changes:
+    - d/p/manpage-debug-packages.patch: mention the -debug package split
+      in Ubuntu systems
+    - libpmemobj c++ bindings are on separate source now:
+      + d/libpmemobj-doc.install: removed
+      + d/libpmemobj-doc.doc-base: removed
+      + d/control: setup transition from libpmemobj-doc to libpmemobj-cpp-doc
+    - d/rules: use dh_missing with --fail and known exclusions for files
+      installed by other dh tools
+    - d/NEWS: added to explain important packaging changes in this update
+      [Can be dropped after 20.04]
+    - d/control: the valgrind test is failing, remove it from
+      build-depends
+    - d/README.source: explains some lintian checks and the reason why
+      jemalloc is bundled.
+    - d/not-installed: list files from the pmreorder python script that
+      we are choosing to not install because it's not ready yet, requiring
+      forks of other tools.
+  * Dropped:
+    - d/rules: don't remove TEST34 as it is passing in Ubuntu atm
+      [In 1.6~rc2-1]
+
+ -- Andreas Hasenack <[e-mail]>  Thu, 23 May 2019 11:57:43 -0300
+
 pmdk (1.6-1) experimental; urgency=medium
 
   * New upstream release.
@@ -10,6 +36,28 @@ pmdk (1.6~rc2-1) experimental; urgency=medium
 
  -- Adam Borowski <[e-mail]>  Mon, 18 Mar 2019 02:04:47 +0100
 
+pmdk (1.5.1-1ubuntu1) eoan; urgency=medium
+
+  * d/p/manpage-debug-packages.patch: mention the -debug package split
+    in Ubuntu systems
+  * libpmemobj c++ bindings are on separate source now:
+    - d/libpmemobj-doc.install: removed
+    - d/libpmemobj-doc.doc-base: removed
+    - d/control: setup transition from libpmemobj-doc to libpmemobj-cpp-doc
+  * d/rules: use dh_missing with --fail and known exclusions for files
+    installed by other dh tools
+  * d/rules: don't remove TEST34 as it is passing in Ubuntu atm
+  * d/NEWS: added to explain important packaging changes in this update
+  * d/control: the valgrind test is failing, remove it from
+    build-depends
+  * d/README.source: explains some lintian checks and the reason why
+    jemalloc is bundled.
+  * d/not-installed: list files from the pmreorder python script that
+    we are choosing to not install because it's not ready yet, requiring
+    forks of other tools.
+
+ -- Andreas Hasenack <[e-mail]>  Thu, 16 May 2019 14:48:30 +0000
+
 pmdk (1.5.1-1) unstable; urgency=medium
 
   * New upstream release.
diff --git a/debian/control b/debian/control
index f23f3dd..6721bf7 100644
--- a/debian/control
+++ b/debian/control
@@ -1,7 +1,8 @@
 Source: pmdk
 Section: libs
 Priority: optional
-Maintainer: Adam Borowski <[e-mail]>
+Maintainer: Ubuntu Developers <[e-mail]>
+XSBC-Original-Maintainer: Adam Borowski <[e-mail]>
 Build-Depends: bash-completion,
                bc,
                debhelper (>= 11),
@@ -11,8 +12,7 @@ Build-Depends: bash-completion,
                libndctl-dev (>= 60.1),
                pandoc,
                pkg-config,
-               strace,
-               valgrind,
+               strace
 Standards-Version: 4.3.0
 Homepage: http://pmem.io/pmdk/
 Vcs-Git: https://github.com/kilobyte/pmdk -b debian
@@ -179,6 +179,14 @@ Description: development files for libpmemobj1
  This package contains libraries and header files used for linking programs
  against libpmemobj.
 
+Package: libpmemobj-doc
+Section: oldlibs
+Architecture: all
+Depends: ${misc:Depends},
+         libpmemobj-cpp-doc
+Description: Transitional package for libpmemobj-cpp-doc
+ This is a transitional package. It can safely be removed.
+
 Package: libpmempool1
 Architecture: amd64
 Depends: libpmem1 (= ${binary:Version}), ${misc:Depends}, ${shlibs:Depends}
diff --git a/debian/libpmemobj-doc.doc-base b/debian/libpmemobj-doc.doc-base
deleted file mode 100644
index 7c6d5fd..0000000
--- a/debian/libpmemobj-doc.doc-base
+++ /dev/null
@@ -1,9 +0,0 @@
-Document: libpmemobj
-Title: PMDK libpmemobj C++ bindings Manual
-Author: PMDK Developers
-Abstract: This is the HTML docs for the C++ bindings for PMDK's libpmemobj.
-Section: Programming
-
-Format: HTML
-Index: /usr/share/doc/libpmemobj-doc/html/index.html
-Files: /usr/share/doc/libpmemobj-doc/html/*
diff --git a/debian/libpmemobj-doc.install b/debian/libpmemobj-doc.install
deleted file mode 100644
index 4a91299..0000000
--- a/debian/libpmemobj-doc.install
+++ /dev/null
@@ -1 +0,0 @@
-usr/share/doc/libpmemobj-doc/*
diff --git a/debian/not-installed b/debian/not-installed
new file mode 100644
index 0000000..d550e92
--- /dev/null
+++ b/debian/not-installed
@@ -0,0 +1,14 @@
+usr/share/pmreorder/utils.py
+usr/share/pmreorder/memoryoperations.py
+usr/share/pmreorder/pmreorder.py
+usr/share/pmreorder/opscontext.py
+usr/share/pmreorder/statemachine.py
+usr/share/pmreorder/consistencycheckwrap.py
+usr/share/pmreorder/loggingfacility.py
+usr/share/pmreorder/operationfactory.py
+usr/share/pmreorder/reorderengines.py
+usr/share/pmreorder/reorderexceptions.py
+usr/share/pmreorder/binaryoutputhandler.py
+usr/share/pmreorder/markerparser.py
+usr/share/bash-completion/completions/pmempool
+usr/bin/pmreorder
diff --git a/debian/patches/manpage-debug-packages.patch b/debian/patches/manpage-debug-packages.patch
new file mode 100644
index 0000000..75a9891
--- /dev/null
+++ b/debian/patches/manpage-debug-packages.patch
@@ -0,0 +1,127 @@
+Description: Mention the -debug package split in Ubuntu systems
+ In Ubuntu systems, the special debug libraries (with extra assertions,
+ checks and logging, not suitable for production) are placed in a separate
+ binary package with the "-debug" suffix. These are only meant to be
+ installed when specific troubleshooting is required. Instructions for this
+ troubleshooting are included in the library manpages, and we add a note
+ explaining that the respective -debug package should be installed.
+Author: Andreas Hasenack <[e-mail]>
+Forwarded: no
+Last-Update: 2019-05-10
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+--- a/doc/libpmem/libpmem.7.md
++++ b/doc/libpmem/libpmem.7.md
+@@ -261,6 +261,10 @@
+ and causing the specified address to be used as a hint about where to
+ place the mapping.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # DEBUGGING AND ERROR HANDLING #
+ 
+@@ -320,6 +324,10 @@
+ the log file is created. If **PMEM_LOG_FILE** is not set, output is
+ written to *stderr*.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/libpmemblk/libpmemblk.7.md
++++ b/doc/libpmemblk/libpmemblk.7.md
+@@ -257,6 +257,10 @@
+ See also **libpmem**(7) for information on other environment variables
+ that may affect **libpmemblk** behavior.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/libpmemlog/libpmemlog.7.md
++++ b/doc/libpmemlog/libpmemlog.7.md
+@@ -256,6 +256,10 @@
+ See also **libpmem**(7) for information about other environment
+ variables affecting **libpmemlog** behavior.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/libpmemobj/libpmemobj.7.md
++++ b/doc/libpmemobj/libpmemobj.7.md
+@@ -258,6 +258,10 @@
+ See also **libpmem**(7) to get information
+ about other environment variables affecting **libpmemobj** behavior.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/libpmempool/libpmempool.7.md
++++ b/doc/libpmempool/libpmempool.7.md
+@@ -214,6 +214,10 @@
+ the log file is created. If **PMEMPOOL_LOG_FILE** is not set, output is
+ written to *stderr*.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/librpmem/librpmem.7.md
++++ b/doc/librpmem/librpmem.7.md
+@@ -355,6 +355,10 @@
+ be appended to the file name when the log file is created. If
+ **RPMEM_LOG_FILE** is not set, logging output is written to *stderr*.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/libvmem/libvmem.7.md
++++ b/doc/libvmem/libvmem.7.md
+@@ -235,6 +235,10 @@
+ the log file is created. If **VMEM_LOG_FILE** is not set, output is
+ written to *stderr*.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # EXAMPLE #
+ 
+--- a/doc/libvmmalloc/libvmmalloc.7.md
++++ b/doc/libvmmalloc/libvmmalloc.7.md
+@@ -255,6 +255,10 @@
+ Setting **VMMALLOC_LOG_STATS** to 1 enables logging human-readable
+ summary statistics at program termination.
+ 
++>NOTE:
++On Ubuntu systems, this extra debug version of the library is
++shipped in the respective **-debug** Debian package and placed in
++the **/usr/lib/$ARCH/pmdk_dbg/** directory.
+ 
+ # NOTES #
+ 
diff --git a/debian/patches/series b/debian/patches/series
index d66650e..e542ea9 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1 +1,2 @@
+manpage-debug-packages.patch
 do-not-build-examples.patch
diff --git a/debian/rules b/debian/rules
index 10638d5..4043d3f 100755
--- a/debian/rules
+++ b/debian/rules
@@ -23,6 +23,9 @@ override_dh_install:
 override_dh_installexamples:
 	dh_installexamples --exclude=.gitignore --exclude=.vcxproj
 
+override_dh_missing:
+	dh_missing --fail-missing --exclude=usr/share/man --exclude=usr/lib/$(DEB_HOST_MULTIARCH)/pmdk_debug/ --exclude=usr/share/bash-completion/completions/pmempool
+
 override_dh_auto_test:
 ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
 	# Use fake pmem; we really want tmpfs if possible.