Merge ~mkukri/ubuntu/+source/presage:merge into ubuntu/+source/presage:debian/sid

[Mate Kukri]

Merge with Debian sid

[Simon Quigley (tsimonq2)]

A few items here that need fixing:
 - The changelog should have another once-over on it. Don't modify *past* changelog entries, but just double check it's complete. I know we discussed some of these elements over IRC, so maybe you're able to just check this one off.
 - We should really think about whether d/p/python.diff should still be in the delta. Is there now support for Python 3 in the Debian package, and we just need to pull that in? Please consider this, and share your thoughts.
 - Generally speaking, I really do like to see proper DEP-3 headers[1]. Lines like these:

+From: Ubuntu Developers <email address hidden>
+Date: Tue, 28 Nov 2023 11:12:03 +0000
+Subject: python

...really aren't verbose enough, and do not contain the proper formatting. An easy way to add a DEP-3 header to the existing patch on the stack is to remove the existing header, then run `quilt header --dep3 -e` which will give you a template to work with.
 - Does this build? Does it pass its autopkgtests? Could you provide more details on this?
 - Having different patch extensions (.diff and .patch) is somewhat annoying, but not a blocker. Please fix this (I *personally* prefer .patch).
 - I would slightly reword this, given that we're a bit past GCC-11: "d/p/c++17.patch: fix the build against GCC-11 by porting the code to C++17 (LP: 1988196)"

Thanks in advance, I look forward to reviewing your followup MP! (You could just re-subscribe sponsors to this one.)

[1] https://dep-team.pages.debian.net/deps/dep3/

review: Needs Fixing

[Mate Kukri (mkukri)]

Hmm I think all the Python related patches can be purged because the last version that built any of them was shipped in focal.

Upon investigation, these touch files that were part of the python-presage and presage-dbus packages that aren't even built anymore. It seems the last merge unnecessarily left these in.

> A few items here that need fixing:
> - The changelog should have another once-over on it. Don't modify *past*
> changelog entries, but just double check it's complete. I know we discussed
> some of these elements over IRC, so maybe you're able to just check this one
> off.
> - We should really think about whether d/p/python.diff should still be in the
> delta. Is there now support for Python 3 in the Debian package, and we just
> need to pull that in? Please consider this, and share your thoughts.
> - Generally speaking, I really do like to see proper DEP-3 headers[1]. Lines
> like these:
>
> +From: Ubuntu Developers <email address hidden>
> +Date: Tue, 28 Nov 2023 11:12:03 +0000
> +Subject: python
>
> ...really aren't verbose enough, and do not contain the proper formatting. An
> easy way to add a DEP-3 header to the existing patch on the stack is to remove
> the existing header, then run `quilt header --dep3 -e` which will give you a
> template to work with.
> - Does this build? Does it pass its autopkgtests? Could you provide more
> details on this?
> - Having different patch extensions (.diff and .patch) is somewhat annoying,
> but not a blocker. Please fix this (I *personally* prefer .patch).
> - I would slightly reword this, given that we're a bit past GCC-11:
> "d/p/c++17.patch: fix the build against GCC-11 by porting the code to C++17
> (LP: 1988196)"
>
> Thanks in advance, I look forward to reviewing your followup MP! (You could
> just re-subscribe sponsors to this one.)
>
> [1] https://dep-team.pages.debian.net/deps/dep3/

[Mate Kukri (mkukri)]

- Dropped Python specific patches and added rationale to changelog:
  python-presage and pyprompter aren't even built anymore so these are not useful.
- Added DEP-3 headers to the Ubuntu patches we still do carry.
- PPA testbuild at: https://launchpad.net/~mkukri/+archive/ubuntu/dev/+build/27047399

[Andreas Hasenack (ahasenack)]

Thanks for addressing those comments, and reducing the delta.

For future merges, please also remember to push the logical/, reconstruct/ and split/ tags, as they help with the review of the delta changes.

+1

review: Approve

[Andreas Hasenack (ahasenack)]

Uploaded:
Uploading presage_0.9.1-2.6ubuntu1.dsc
Uploading presage_0.9.1-2.6ubuntu1.debian.tar.xz
Uploading presage_0.9.1-2.6ubuntu1_source.buildinfo
Uploading presage_0.9.1-2.6ubuntu1_source.changes

diff --git a/debian/changelog b/debian/changelog
index 38aafa0..e779147 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,25 @@
+presage (0.9.1-2.6ubuntu1) noble; urgency=medium
+
+  * Merge with Debian unstable. Remaining changes:
+    - d/p/dont-tokenize-on-apostrophes.patch: Allow words with
+      apostrophes to be predicted
+    - d/p/add_presage_config_environment_variable.patch:
+      Replace use_presage_prefix_path_in_default_profile.patch with a new
+      patch making sure all works in snap environments
+    - d/p/add-missing-online-learning-configuration-value.patch:
+      Add missing ONLINE_LEARNING configuration value to defaultProfile
+  * Patches now included in Debian:
+    - d/p/c++17.patch: fix the build with C++17 compliant compilers
+      (LP: 1988196)
+    - d/p/format-security.patch: fix insecure format strings in the demo code
+      (FTBFS on Ubuntu due to -Werror=format-security)
+  * Drop patches for python-presage and pyprompter (no longer being built):
+    - d/p/ubuntu_dont_require_wxpython.patch: Do not check for wx
+      to enable pyprompter
+    - d/p/python.diff: Explicitly use python2 instead of python in the build.
+
+ -- Mate Kukri <[e-mail]>  Tue, 28 Nov 2023 11:07:48 +0000
+
 presage (0.9.1-2.6) unstable; urgency=medium
 
   * Non-maintainer upload.
@@ -63,6 +85,41 @@ presage (0.9.1-2.3) unstable; urgency=medium
 
  -- Nilesh Patra <[e-mail]>  Fri, 14 Oct 2022 08:39:51 +0530
 
+presage (0.9.1-2.2ubuntu3) kinetic; urgency=medium
+
+  * d/p/c++17.patch: fix the build against GCC-11 by porting the code to C++17
+    (LP: #1988196)
+  * d/p/format-security.patch: fix insecure format strings in the demo code
+    (FTBFS on Ubuntu due to -Werror=format-security.patch)
+
+ -- Simon Chopin <[e-mail]>  Tue, 30 Aug 2022 17:56:09 +0200
+
+presage (0.9.1-2.2ubuntu2) jammy; urgency=medium
+
+  * No-change rebuild against latest gdk-pixbuf
+
+ -- Jeremy Bicha <[e-mail]>  Wed, 09 Mar 2022 18:53:51 -0500
+
+presage (0.9.1-2.2ubuntu1) groovy; urgency=low
+
+  * Merge from Debian unstable.  Remaining changes:
+    - Remove python-wxgtk3.0 from Build-Depends
+    - debian/patches/ubuntu_dont_require_wxpython.patch: Do not check for wx
+      to enable pyprompter
+    - debian/patches/dont-tokenize-on-apostrophes.patch: Allow words with
+      apostrophes to be predicted
+    - debian/patches/use_presage_prefix_path_in_default_profile.patch:
+      Use the PRESAGE_PREFIX_PATH environment variable in the default
+      profile
+    - debian/patches/add-missing-online-learning-configuration-value.patch:
+      Add missing ONLINE_LEARNING configuration value to defaultProfile
+    - debian/patches/add_presage_config_environment_variable.patch:
+      Replace use_presage_prefix_path_in_default_profile.patch with a new
+      patch making sure all works in snap environments
+    - Explicitly use python2 instead of python in the build.
+
+ -- Steve Langasek <[e-mail]>  Wed, 06 May 2020 14:01:16 -0700
+
 presage (0.9.1-2.2) unstable; urgency=medium
 
   [ Aron Xu ]
@@ -75,6 +132,65 @@ presage (0.9.1-2.2) unstable; urgency=medium
 
  -- Aron Xu <[e-mail]>  Sat, 25 Jan 2020 09:31:19 +0000
 
+presage (0.9.1-2.1ubuntu9) focal; urgency=medium
+
+  * No-change rebuild with fixed binutils on arm64.
+
+ -- Matthias Klose <[e-mail]>  Sat, 08 Feb 2020 11:17:25 +0000
+
+presage (0.9.1-2.1ubuntu8) focal; urgency=medium
+
+  * No-change rebuild to generate dependencies on python2.
+
+ -- Matthias Klose <[e-mail]>  Sat, 11 Jan 2020 18:56:52 +0000
+
+presage (0.9.1-2.1ubuntu7) focal; urgency=medium
+
+  * Explicitly use python2 instead of python in the build.
+
+ -- Matthias Klose <[e-mail]>  Thu, 09 Jan 2020 15:50:36 +0100
+
+presage (0.9.1-2.1ubuntu6) focal; urgency=medium
+
+  * No-change rebuild to drop dependencies on python2.
+
+ -- Matthias Klose <[e-mail]>  Thu, 09 Jan 2020 13:50:47 +0100
+
+presage (0.9.1-2.1ubuntu5) cosmic; urgency=medium
+
+  * No-change rebuild for ncurses soname changes.
+
+ -- Matthias Klose <[e-mail]>  Thu, 03 May 2018 14:18:10 +0000
+
+presage (0.9.1-2.1ubuntu4) zesty; urgency=medium
+
+  * debian/patches/add_presage_config_environment_variable.patch:
+    - Replace use_presage_prefix_path_in_default_profile.patch with a new patch
+      making sure all works in snap environments
+
+ -- Łukasz 'sil2100' Zemczak <[e-mail]>  Thu, 08 Dec 2016 10:54:00 +0100
+
+presage (0.9.1-2.1ubuntu3) zesty; urgency=medium
+
+  [ Michael Sheldon ]
+  * debian/patches/use_presage_prefix_path_in_default_profile.patch:
+    - Use the PRESAGE_PREFIX_PATH environment variable in the default profile
+  * debian/patches/add-missing-online-learning-configuration-value.patch:
+    - Add missing ONLINE_LEARNING configuration value to defaultProfile
+
+ -- Łukasz 'sil2100' Zemczak <[e-mail]>  Fri, 04 Nov 2016 18:05:23 +0100
+
+presage (0.9.1-2.1ubuntu1) yakkety; urgency=medium
+
+  * Merge with Debian; remaining changes:
+    - Remove python-wxgtk3.0 from Build-Depends
+    - debian/patches/ubuntu_dont_require_wxpython.patch: Do not check for wx
+      to enable pyprompter
+    - debian/patches/dont-tokenize-on-apostrophes.patch: Allow words with
+      apostrophes to be predicted
+
+ -- Matthias Klose <[e-mail]>  Tue, 06 Sep 2016 13:56:40 +0200
+
 presage (0.9.1-2.1) unstable; urgency=medium
 
   * Non-maintainer upload.
@@ -82,6 +198,17 @@ presage (0.9.1-2.1) unstable; urgency=medium
 
  -- Aron Xu <[e-mail]>  Sun, 03 Jul 2016 20:56:47 +0800
 
+presage (0.9.1-2ubuntu1) yakkety; urgency=medium
+
+  * Merge with Debian; remaining changes:
+    - Remove python-wxgtk3.0 from Build-Depends
+    - debian/patches/ubuntu_dont_require_wxpython.patch: Do not check for wx
+      to enable pyprompter
+    - debian/patches/dont-tokenize-on-apostrophes.patch: Allow words with
+      apostrophes to be predicted
+
+ -- Matthias Klose <[e-mail]>  Thu, 19 May 2016 15:17:23 +0200
+
 presage (0.9.1-2) unstable; urgency=medium
 
   * Depend on swig instead of swig2.0 (Closes: #816999)
@@ -93,6 +220,29 @@ presage (0.9.1-2) unstable; urgency=medium
 
  -- Matteo Vescovi <[e-mail]>  Thu, 17 Mar 2016 13:06:19 +0100
 
+presage (0.9.1-1.1ubuntu2) xenial; urgency=medium
+
+  * No-change rebuild for ncurses6 transition.
+
+ -- Matthias Klose <[e-mail]>  Sun, 07 Feb 2016 09:45:05 +0000
+
+presage (0.9.1-1.1ubuntu1) xenial; urgency=low
+
+  * Merge from Debian unstable.  Remaining changes:
+    - debian/patches/lib_underlinkage.patch: Added missing lib to link,
+      fixing FTBFS.
+    - Remove python-wxgtk3.0 from Build-Depends
+    - debian/patches/ubuntu_dont_require_wxpython.patch: Do not check for wx
+      to enable pyprompter
+    - debian/patches/dont-tokenize-on-apostrophes.patch: Allow words with
+      apostrophes to be predicted
+    - Build-depend on swig (3.0) instead of swig2.0.
+  * Dropped changes, included in Debian:
+    - Build using GCC 5.
+    - Update symbols file.
+
+ -- Steve Langasek <[e-mail]>  Sat, 31 Oct 2015 22:10:08 -0700
+
 presage (0.9.1-1.1) unstable; urgency=medium
 
   * Non-maintainer upload.
@@ -107,6 +257,52 @@ presage (0.9.1-1) unstable; urgency=medium
 
  -- Aron Xu <[e-mail]>  Sat, 15 Aug 2015 23:44:45 +0800
 
+presage (0.9.1-0ubuntu3) xenial; urgency=medium
+
+  * Build-depend on swig (3.0) instead of swig2.0.
+
+ -- Steve Langasek <[e-mail]>  Fri, 30 Oct 2015 13:09:52 -0700
+
+presage (0.9.1-0ubuntu2) wily; urgency=medium
+
+  * Build using GCC 5.
+
+ -- Matthias Klose <[e-mail]>  Wed, 15 Jul 2015 20:21:30 +0200
+
+presage (0.9.1-0ubuntu1) wily; urgency=medium
+
+  * New upstream version.
+    - No test failures with GCC 5.
+  * Update symbols file.
+
+ -- Matthias Klose <[e-mail]>  Wed, 15 Jul 2015 20:09:44 +0200
+
+presage (0.9-1ubuntu3) vivid; urgency=medium
+
+  * Remove python-wxgtk3.0 from Build-Depends (LP: #1430923)
+  * debian/patches/ubuntu_dont_require_wxpython.patch:
+    - Do not check for wx to enable pyprompter
+
+ -- Aron Xu <[e-mail]>  Thu, 12 Mar 2015 00:38:45 +0800
+
+presage (0.9-1ubuntu2) vivid; urgency=medium
+
+  * debian/patches/dont-tokenize-on-apostrophes.patch:
+    - Allow words with apostrophes to be predicted (lp: #1384800)
+
+ -- Michael Sheldon <[e-mail]>  Wed, 04 Feb 2015 10:16:01 +0100
+
+presage (0.9-1ubuntu1) vivid; urgency=low
+
+  * Merge from Debian unstable.  Remaining changes:
+    - debian/patches/lib_underlinkage.patch:
+      + Added missing lib to link, fixing FTBFS. (Closes: #776720)
+  * Drop following patches, fixed upstream:
+    - debian/patches/automake-parallel.patch
+    - debian/patches/no-automake-werror.patch
+
+ -- Artur Rona <[e-mail]>  Sat, 31 Jan 2015 20:19:30 +0100
+
 presage (0.9-1) unstable; urgency=medium
 
   * New upstream release.
@@ -135,6 +331,28 @@ presage (0.8.9-1) unstable; urgency=low
 
  -- Matteo Vescovi <[e-mail]>  Wed, 02 OCt 2013 18:23:01 +0100
 
+presage (0.8.8-1ubuntu3) trusty; urgency=medium
+
+  * Use dh-autoreconf to update libtool for ppc64el.
+  * Disable automake -Werror.
+  * Make shell based test script compatible with automakes (new) default
+    parallel test runner.
+
+ -- Dimitri John Ledkov <[e-mail]>  Tue, 24 Dec 2013 17:28:18 +0000
+
+presage (0.8.8-1ubuntu2) trusty; urgency=low
+
+  * Convert libpresage1, libpresage1-dbg, libpresage-data, and
+    libpresage-dev binary packages to multiarch.
+
+ -- Colin Watson <[e-mail]>  Thu, 07 Nov 2013 14:19:59 +0000
+
+presage (0.8.8-1ubuntu1) raring; urgency=low
+
+  * Added missing lib to link, fixing FTBFS.
+
+ -- Daniel T Chen <[e-mail]>  Thu, 18 Apr 2013 16:10:04 -0400
+
 presage (0.8.8-1) unstable; urgency=low
 
   * New upstream release.
diff --git a/debian/control b/debian/control
index d8e69a5..671afda 100644
--- a/debian/control
+++ b/debian/control
@@ -1,7 +1,8 @@
 Source: presage
 Section: devel
 Priority: optional
-Maintainer: Matteo Vescovi <[e-mail]>
+Maintainer: Ubuntu Developers <[e-mail]>
+XSBC-Original-Maintainer: Matteo Vescovi <[e-mail]>
 Build-Depends: debhelper-compat (= 13),
  pkg-config,
  libsqlite3-dev,
diff --git a/debian/patches/add-missing-online-learning-configuration-value.patch b/debian/patches/add-missing-online-learning-configuration-value.patch
new file mode 100644
index 0000000..199747d
--- /dev/null
+++ b/debian/patches/add-missing-online-learning-configuration-value.patch
@@ -0,0 +1,31 @@
+Description: Add missing ONLINE_LEARNING configuration value to defaultProfile
+ Fixes a crash in libpresage when a configuration file isn't found.
+Author: Michael Sheldon <[e-mail]>
+Bug: https://sourceforge.net/p/presage/patches/3/
+Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/presage/+bug/1639251
+Forwarded: https://sourceforge.net/p/presage/patches/3/
+Last-Update: 2023-12-04
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+diff --git a/src/lib/core/defaultProfile.cpp b/src/lib/core/defaultProfile.cpp
+index fe0d9bd..1f7e365 100644
+--- a/src/lib/core/defaultProfile.cpp
++++ b/src/lib/core/defaultProfile.cpp
+@@ -62,7 +62,16 @@ void DefaultProfile::build_xml_document (const std::string& filename)
+ "        <!-- LOWERCASE_MODE"
+ "          Instruct context tracker to track text as lowercase"
+ "        -->"
+-"         <LOWERCASE_MODE>yes</LOWERCASE_MODE>"
++"        <LOWERCASE_MODE>yes</LOWERCASE_MODE>"
++"        <!-- ONLINE_LEARNING"
++"          Controls presage online machine learning feature."
++"          Presage is context-aware and capable of dynamic online learning."
++"          Setting this to yes/true will enable online learning mode."
++"          Setting this to no/false will disable online learning mode."
++"          When online learning mode is disabled, it is still"
++"          possible to instruct presage to learn through its API."
++"        -->"
++"        <ONLINE_LEARNING>yes</ONLINE_LEARNING>"
+ "    </ContextTracker>"
+ "    <Selector>"
+ "        <LOGGER>ERROR</LOGGER>"
diff --git a/debian/patches/add_presage_config_environment_variable.patch b/debian/patches/add_presage_config_environment_variable.patch
new file mode 100644
index 0000000..0809edd
--- /dev/null
+++ b/debian/patches/add_presage_config_environment_variable.patch
@@ -0,0 +1,35 @@
+Description: Add PRESAGE_CONFIG environment variable
+ Provide an environment variable for setting the location that presage should
+ check for its system-wide configuration file. This allows for usage within
+ snap packages.
+Author: Michael Sheldon <[e-mail]>
+Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/presage/+bug/1639270
+Forwarded: no
+Last-Update: 2023-12-04
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+diff --git a/src/lib/core/profileManager.cpp b/src/lib/core/profileManager.cpp
+index a17809d..3ded80c 100644
+--- a/src/lib/core/profileManager.cpp
++++ b/src/lib/core/profileManager.cpp
+@@ -121,10 +121,16 @@ void ProfileManager::init_profiles (const std::string& profilename)
+     }
+ 
+ 
+-    // system etc directory
+-    profiles.push_back (get_system_etc_dir() + '/' + DefaultProfile::DEFAULT_PROFILE_FILENAME);
+-    // installation config directory
+-    profiles.push_back (static_cast<std::string>(sysconfdir) + '/' + DefaultProfile::DEFAULT_PROFILE_FILENAME);
++    // config specified by environment overrides system config
++    char *env_profile = getenv("PRESAGE_CONFIG");
++    if (env_profile != NULL) {
++        profiles.push_back(env_profile);
++    } else {
++        // system etc directory
++        profiles.push_back (get_system_etc_dir() + '/' + DefaultProfile::DEFAULT_PROFILE_FILENAME);
++        // installation config directory
++        profiles.push_back (static_cast<std::string>(sysconfdir) + '/' + DefaultProfile::DEFAULT_PROFILE_FILENAME);
++    }
+     // home dir dotfile
+     profiles.push_back (get_user_home_dir() + "/.presage/" + DefaultProfile::DEFAULT_PROFILE_FILENAME);
+     // user specified profile (if any)
diff --git a/debian/patches/dont-tokenize-on-apostrophes.patch b/debian/patches/dont-tokenize-on-apostrophes.patch
new file mode 100644
index 0000000..8425dec
--- /dev/null
+++ b/debian/patches/dont-tokenize-on-apostrophes.patch
@@ -0,0 +1,63 @@
+Description: Allow words with apostrophes to be predicted
+ Stop the tokenizer from splitting based on apostrophes and allow for the
+ escaping of words containing apostrophes in the database connector.
+Author: Michael Sheldon <[e-mail]>
+Bug: https://sourceforge.net/p/presage/patches/2/
+Bug-Ubuntu: https://launchpad.net/bugs/1384800
+Bug-Debian: https://bugs.debian.org/770831
+Forwarded: https://sourceforge.net/p/presage/patches/2/
+Last-Update: 2023-12-04
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+diff --git a/src/lib/core/charsets.h b/src/lib/core/charsets.h
+index 1c74d76..c1a6bc0 100644
+--- a/src/lib/core/charsets.h
++++ b/src/lib/core/charsets.h
+@@ -180,7 +180,6 @@ const char DEFAULT_SEPARATOR_CHARS[]={
+     '$',
+     '%',
+     '&',
+-    '\'',
+     '(',
+     ')',
+     '*',
+diff --git a/src/lib/predictors/dbconnector/databaseConnector.cpp b/src/lib/predictors/dbconnector/databaseConnector.cpp
+index 15de52a..65cd629 100644
+--- a/src/lib/predictors/dbconnector/databaseConnector.cpp
++++ b/src/lib/predictors/dbconnector/databaseConnector.cpp
+@@ -293,12 +293,16 @@ std::string DatabaseConnector::buildValuesClause(const Ngram ngram, const int co
+ 
+ std::string DatabaseConnector::sanitizeString(const std::string str) const
+ {
+-    // TODO
+-    // just return the string for the time being
+-    // REVISIT
+-    // TO BE DONE
+-    // TBD
+-    return str;
++    // Escape single quotes
++    std::size_t searchPos = 0;
++    std::size_t pos;
++    std::string result(str);
++    while((pos = result.find("'", searchPos)) != std::string::npos)
++    {
++          result.replace(pos, 1, "''");
++          searchPos = pos + 2;
++    }
++    return result;
+ }
+ 
+ int DatabaseConnector::extractFirstInteger(const NgramTable& table) const
+diff --git a/src/tools/text2ngram.cpp b/src/tools/text2ngram.cpp
+index 807aa83..20a59f7 100644
+--- a/src/tools/text2ngram.cpp
++++ b/src/tools/text2ngram.cpp
+@@ -174,7 +174,7 @@ int main(int argc, char* argv[])
+ 	std::ifstream infile(argv[i]);
+ 	ForwardTokenizer tokenizer(infile,
+ 				   " \f\n\r\t\v ",
+-				   "`~!@#$%^&*()_-+=\\|]}[{'\";:/?.>,<«»");
++				   "`~!@#$%^&*()_-+=\\|]}[{\";:/?.>,<«»");
+ 	tokenizer.lowercaseMode(lowercase);
+ 
+ 	// take care of first N-1 tokens
diff --git a/debian/patches/series b/debian/patches/series
index 04b6515..29084a0 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -4,3 +4,6 @@ format-security.patch
 sfos/docs-Install-css-too.patch
 sfos/text2ngram-Add-and-to-separators.patch
 sfos/text2ngram-Add-non-breaking-space-to-whitespace-chars.patch
+dont-tokenize-on-apostrophes.patch
+add_presage_config_environment_variable.patch
+add-missing-online-learning-configuration-value.patch