Merge lp:~john+ubuntu-g/openlp/singingthefaith into lp:openlp

Proposed by John Lines on 2019-06-30
Status: Superseded
Proposed branch: lp:~john+ubuntu-g/openlp/singingthefaith
Merge into: lp:openlp
Diff against target: 633 lines (+549/-11)
8 files modified
openlp/plugins/songs/lib/importer.py (+22/-10)
openlp/plugins/songs/lib/importers/singingthefaith.py (+396/-0)
tests/functional/openlp_plugins/songs/test_singingthefaithimport.py (+48/-0)
tests/helpers/songfileimport.py (+2/-1)
tests/resources/songs/singingthefaith/H1.txt (+9/-0)
tests/resources/songs/singingthefaith/H2.txt (+30/-0)
tests/resources/songs/singingthefaith/STF001.json (+13/-0)
tests/resources/songs/singingthefaith/STF002.json (+29/-0)
To merge this branch: bzr merge lp:~john+ubuntu-g/openlp/singingthefaith
Reviewer Review Type Date Requested Status
Tomas Groth Needs Fixing on 2019-07-14
Phill Needs Fixing on 2019-07-03
Raoul Snyman 2019-06-30 Pending
Review via email: mp+369490@code.launchpad.net

This proposal supersedes a proposal from 2019-06-30.

This proposal has been superseded by a proposal from 2019-07-19.

Commit message

Initial merge of SingingTheFaithImport, including update to importer.py

Description of the change

Singing The Faith is the new Authorized Hymn book for the Methodist Church of Great Britain.
There is an electronic version of the Hymn book, for Windows only, which can export Hymns as text files.

This import module smooths the process of converting these text files into OpenLP. The input format is messy and not intended for automatic processing so the importer uses a combination of heuristics and a hints file. This version has not been tested on all the hymns in Singing The Faith, but deals with most of the, more than 100, hymns it has been tested with.

Note that it includes a test module, which works for the single verse case. Multiple verse songs import OK, but tests fail.

To post a comment you must log in.
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests failed, please see https://ci.openlp.io/job/MP-02-Linux_Tests/191/ for more details

Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

The test is failing because the "add_verse" method is not called, or not called with that exact data. I've broken up the error message into 3 parts below for easier reading.

AssertionError:

add_verse('Amazing grace! How sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.', 'v1')

call not found

review: Needs Fixing
Phill (phill-ridout) wrote : Posted in a previous version of this proposal

A few in-line comments. Also you don't need to use parenthesis around the expressions in the if statements.

John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

> A few in-line comments. Also you don't need to use parenthesis around the
> expressions in the if statements.

Thanks - have updated to use Path more, and have removed redudant parentheses round expressions in if statements

Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Hey John, a couple things.

1. You have a ton of linting issues. I started commenting, and then I realised that it would be better to just point you to flake8. See the link at the bottom of my comment for a quick introduction to linting and flake8.

2. You have some inconsistent indentation. Indentation in Python is very important, so we take it seriously. Also, please make sure you are indenting using spaces and not tabs.

3. You are not committing your code with the e-mail address associated with Launchpad. Please can you fix that by issuing a bzr whoami "John Lines <email address hidden>"

4. Once you've made all your changes, and you're ready for another review, you need to resubmit your merge proposal. Do this by clicking the "Resubmit" link in the top right hand corner of the page.

5. If you're struggling with anything, pop into our IRC channel, there's usually someone around who is happy to help.

https://medium.com/python-pandemonium/what-is-flake8-and-why-we-should-use-it-b89bd78073f2

review: Needs Fixing
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

1. Thanks for the info on flake8 - is now flake8 clean - you are quite right - it is better for me to learn about the tools.
2. Think indentation is fixed - flake8 was handy as well
3. Email address now set in my bzr config
4. Can you have another look at the change now.

Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests passed!

Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linting failed, please see https://ci.openlp.io/job/MP-03-Linting/132/ for more details

John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

Fix lint tests outside main importer code

Raoul Snyman (raoul-snyman) wrote :

Linux tests passed!

Raoul Snyman (raoul-snyman) wrote :

Linting passed!

Raoul Snyman (raoul-snyman) wrote :

macOS tests passed!

Phill (phill-ridout) wrote :

Please change your string formatting to use the 'new' style with the format function. ( https://pyformat.info/ ) also string formatting is preferred over concatenation (i.e, "part1" + var + "part2")

single quotes for strings, not double quotes

do_import_file is very long can this be split in to smaller methods?

review: Needs Fixing
2890. By John Lines on 2019-07-12

Minimize differences needed to run under OpenLP 2.4.6

2891. By John Lines on 2019-07-12

Convert double to single quotes, make hint variable names consistent

2892. By John Lines on 2019-07-12

New style string formatting, restructure indent=0 cases

Tomas Groth (tomasgroth) wrote :

Generally it looks good to me, though I haven't tested. But a few things to fix.

review: Needs Fixing
2893. By John Lines on 2019-07-13

New hints for AddSpaceAfterSemi and CCLI

2894. By John Lines on 2019-07-15

Update GPL version, use constructor, add comment to signal unepected verse order

John Lines (john+ubuntu-g) wrote :

Using new string formatting, constructor for class.

2895. By John Lines on 2019-07-16

Add AddSpaceAfterColon hint

2896. By John Lines on 2019-07-17

Add BlankLine hint and deal with a leading asterisk on a verse number

2897. By John Lines on 2019-07-19

Merge trunk updates

2898. By John Lines on 2019-07-19

enable whole song impoerter test

2899. By John Lines on 2019-07-27

Strip unwanted formatting characters

2900. By John Lines on 2019-07-28

Implement BoldLine hint

2901. By John Lines on 2019-07-28

Make Based on Psalm an automatic comment, do not automatically make Authors type Word

2902. By John Lines on 2019-07-28

Fix typo

2903. By John Lines on 2019-08-12

Merge trunk updates

2904. By John Lines on 2019-08-23

Use .format in importer.py

2905. By John Lines on 2019-08-23

Implement SongbookNumberInTitle hint

2906. By John Lines on 2019-08-29

Upload hints.tag into resources

2907. By John Lines on 2019-09-03

fix tests to allow for SongbookNumberInTitle defaulting to false and add hints tests

2908. By John Lines on 2019-09-03

Merge trunk updates

2909. By John Lines on 2019-09-03

add tests with hints subdirectory

2910. By John Lines on 2019-09-03

Linting fix

2911. By John Lines on 2019-09-04

put default hints plugin directory and chnage name to singingthefaith-hints.tag

2912. By John Lines on 2019-09-16

Merge trunk updates

2913. By John Lines on 2019-09-16

Tweaks to hints - now version 3, H470 and H567

Unmerged revisions

2913. By John Lines on 2019-09-16

Tweaks to hints - now version 3, H470 and H567

2912. By John Lines on 2019-09-16

Merge trunk updates

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'openlp/plugins/songs/lib/importer.py'
2--- openlp/plugins/songs/lib/importer.py 2019-04-13 13:00:22 +0000
3+++ openlp/plugins/songs/lib/importer.py 2019-07-19 11:11:28 +0000
4@@ -42,6 +42,7 @@
5 from .importers.powersong import PowerSongImport
6 from .importers.presentationmanager import PresentationManagerImport
7 from .importers.propresenter import ProPresenterImport
8+from .importers.singingthefaith import SingingTheFaithImport
9 from .importers.songbeamer import SongBeamerImport
10 from .importers.songpro import SongProImport
11 from .importers.songshowplus import SongShowPlusImport
12@@ -173,16 +174,17 @@
13 PowerSong = 16
14 PresentationManager = 17
15 ProPresenter = 18
16- SongBeamer = 19
17- SongPro = 20
18- SongShowPlus = 21
19- SongsOfFellowship = 22
20- SundayPlus = 23
21- VideoPsalm = 24
22- WordsOfWorship = 25
23- WorshipAssistant = 26
24- WorshipCenterPro = 27
25- ZionWorx = 28
26+ SingingTheFaith = 19
27+ SongBeamer = 20
28+ SongPro = 21
29+ SongShowPlus = 22
30+ SongsOfFellowship = 23
31+ SundayPlus = 24
32+ VideoPsalm = 25
33+ WordsOfWorship = 26
34+ WorshipAssistant = 27
35+ WorshipCenterPro = 28
36+ ZionWorx = 29
37
38 # Set optional attribute defaults
39 __defaults__ = {
40@@ -343,6 +345,15 @@
41 'filter': '{text} (*.pro4 *.pro5 *.pro6)'.format(text=translate('SongsPlugin.ImportWizardForm',
42 'ProPresenter Song Files'))
43 },
44+ SingingTheFaith: {
45+ 'class': SingingTheFaithImport,
46+ 'name': 'SingingTheFaith',
47+ 'prefix': 'singingTheFaith',
48+ 'filter': '%s (*.txt)' % translate('SongsPlugin.ImportWizardForm', 'Singing The Faith Exported Files'),
49+ 'descriptionText': translate('SongsPlugin.ImportWizardForm',
50+ 'First use Singing The Faith Electonic edition to export '
51+ 'the song(s) in Text format.')
52+ },
53 SongBeamer: {
54 'class': SongBeamerImport,
55 'name': 'SongBeamer',
56@@ -462,6 +473,7 @@
57 SongFormat.PowerSong,
58 SongFormat.PresentationManager,
59 SongFormat.ProPresenter,
60+ SongFormat.SingingTheFaith,
61 SongFormat.SongBeamer,
62 SongFormat.SongPro,
63 SongFormat.SongShowPlus,
64
65=== added file 'openlp/plugins/songs/lib/importers/singingthefaith.py'
66--- openlp/plugins/songs/lib/importers/singingthefaith.py 1970-01-01 00:00:00 +0000
67+++ openlp/plugins/songs/lib/importers/singingthefaith.py 2019-07-19 11:11:28 +0000
68@@ -0,0 +1,396 @@
69+# -*- coding: utf-8 -*-
70+# vim: autoindent shiftwidth=4 expandtab textwidth=120 tabstop=4 softtabstop=4
71+
72+###############################################################################
73+# OpenLP - Open Source Lyrics Projection #
74+# --------------------------------------------------------------------------- #
75+# Copyright (c) 2008-2019 OpenLP Developers #
76+# --------------------------------------------------------------------------- #
77+# This program is free software; you can redistribute it and/or modify it #
78+# under the terms of the GNU General Public License as published by the Free #
79+# Software Foundation; version 3 of the License. #
80+# #
81+# This program is distributed in the hope that it will be useful, but WITHOUT #
82+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or #
83+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for #
84+# more details. #
85+# #
86+# You should have received a copy of the GNU General Public License along #
87+# with this program; if not, write to the Free Software Foundation, Inc., 59 #
88+# Temple Place, Suite 330, Boston, MA 02111-1307 USA #
89+###############################################################################
90+"""
91+The :mod:`singingthefaith` module provides the functionality for importing songs which are
92+exported from Singing The Faith - an Authorised songbook for the Methodist Church of
93+Great Britain."""
94+
95+import logging
96+import re
97+from pathlib import Path
98+
99+from openlp.core.common.i18n import translate
100+from openlp.plugins.songs.lib.importers.songimport import SongImport
101+
102+log = logging.getLogger(__name__)
103+
104+
105+class SingingTheFaithImport(SongImport):
106+ """
107+ Import songs exported from SingingTheFaith
108+ """
109+
110+ def __init__(self, manager, **kwargs):
111+ """
112+ Initialise the class.
113+ """
114+ super(SingingTheFaithImport, self).__init__(manager, **kwargs)
115+ self.hints_available = False
116+ self.checks_needed = True
117+ self.hint_line = {}
118+ self.hint_file_version = '0'
119+ self.hint_verse_order = ''
120+ self.hint_song_title = ''
121+ self.hint_comments = ''
122+ self.hint_ccli = ''
123+ self.hint_ignore_indent = False
124+
125+ def do_import(self):
126+ """
127+ Receive a single file or a list of files to import.
128+ """
129+ if not isinstance(self.import_source, list):
130+ return
131+ self.import_wizard.progress_bar.setMaximum(len(self.import_source))
132+ for file_path in self.import_source:
133+ if self.stop_import_flag:
134+ return
135+ if isinstance(file_path, str):
136+ song_file = open(file_path, 'rt', encoding='cp1251')
137+ self.do_import_file(song_file)
138+ song_file.close()
139+ else:
140+ with file_path.open('rt', encoding='cp1251') as song_file:
141+ self.do_import_file(song_file)
142+
143+ def do_import_file(self, file):
144+ """
145+ Process the SingingTheFaith file - pass in a file-like object, not a file path.
146+ """
147+ singing_the_faith_version = 1
148+ self.set_defaults()
149+ # Setup variables
150+ line_number = 0
151+ old_indent = 0
152+ # The chorus indent is how many spaces the chorus is indented - it might be 6,
153+ # but we test for >= and I do not know how consistent to formatting of the
154+ # exported songs is.
155+ chorus_indent = 5
156+ song_title = 'STF000 -'
157+ song_number = '0'
158+ ccli = '0'
159+ current_verse = ''
160+ current_verse_type = 'v'
161+ current_verse_number = 1
162+ # Potentially we could try to track current chorus number to automatically handle
163+ # more than 1 chorus, currently unused.
164+ # current_chorus_number = 1
165+ has_chorus = False
166+ chorus_written = False
167+ auto_verse_order_ok = False
168+ copyright = ''
169+ # the check_flag is prepended to the title, removed if the import should be OK
170+ # all the songs which need manual editing should sort below all the OK songs
171+ check_flag = 'z'
172+
173+ self.add_comment('Imported with Singing The Faith Importer v ' + str(singing_the_faith_version))
174+
175+ # Get the file_song_number - so we can use it for hints
176+ filename = Path(file.name)
177+ song_number_file = filename.stem
178+ song_number_match = re.search(r'\d+', song_number_file)
179+ if song_number_match:
180+ song_number_file = song_number_match.group()
181+
182+ # See if there is a hints file in the same location as the file
183+ dir_path = filename.parent
184+ hints_file_path = dir_path / 'hints.tag'
185+ try:
186+ with hints_file_path.open('r') as hints_file:
187+ hints_available = self.read_hints(hints_file, song_number_file)
188+ except FileNotFoundError:
189+ hints_available = False
190+
191+ try:
192+ for line in file:
193+ line_number += 1
194+ if hints_available and str(line_number) in self.hint_line:
195+ hint = self.hint_line[str(line_number)]
196+ # Set to false if this hint does not replace the line
197+ line_replaced = True
198+ if hint == 'Comment':
199+ line.strip()
200+ self.add_comment(line)
201+ continue
202+ elif hint == 'Ignore':
203+ continue
204+ elif hint == 'Author':
205+ # add as a raw author - do not split and make them a words author
206+ line.strip()
207+ self.add_author(line, 'words')
208+ line_number += 1
209+ next(file)
210+ continue
211+ elif hint.startswith('VariantVerse'):
212+ vv, hintverse, replace = hint.split(' ', 2)
213+ this_verse = self.verses[int(hintverse) - 1]
214+ this_verse_str = this_verse[1]
215+ new_verse = this_verse_str
216+ # There might be multiple replace pairs separated by |
217+ replaces = replace.split('|')
218+ for rep in replaces:
219+ source_str, dest_str = rep.split('/')
220+ new_verse = new_verse.replace(source_str, dest_str)
221+ self.add_verse(new_verse, 'v')
222+ self.verse_order_list.append('v{}'.format(str(current_verse_number)))
223+ current_verse_number += 1
224+ line_number += 1
225+ next(file)
226+ continue
227+ elif hint == 'AddSpaceAfterSemi':
228+ line = line.replace(';', '; ')
229+ line_replaced = False
230+ # note - do not use contine here as the line should now be processed as normal.
231+ elif hint == 'AddSpaceAfterColon':
232+ line = line.replace(':', ': ')
233+ line_replaced = False
234+ elif hint == 'BlankLine':
235+ line = ' *Blank*'
236+ line_replaced = False
237+ else:
238+ self.log_error(translate('SongsPlugin.SingingTheFaithImport',
239+ 'File {file})'.format(file=file.name)),
240+ translate('SongsPlugin.SingingTheFaithImport',
241+ 'Unknown hint {hint}').format(hint=hint))
242+ if line_replaced:
243+ return
244+ # STF exported lines have a leading verse number at the start of each verse.
245+ # remove them - note that we want to track the indent as that shows a chorus
246+ # so will deal with that before stripping all leading spaces.
247+ indent = 0
248+ if line.strip():
249+ # One hymn has one line which starts '* 6' at the start of a verse
250+ # Strip this out
251+ if line.startswith('* 6'):
252+ line = line.lstrip('* ')
253+ verse_num_match = re.search(r'^\d+', line)
254+ if verse_num_match:
255+ # Could extract the verse number and check it against the calculated
256+ # verse number - TODO
257+ # verse_num = verse_num_match.group()
258+ line = line.lstrip('0123456789')
259+ indent_match = re.search(r'^\s+', line)
260+ if indent_match:
261+ indent = len(indent_match.group())
262+ # Assuming we have sorted out what is verse and what is chorus, strip lines,
263+ # unless ignoreIndent
264+ if self.hint_ignore_indent:
265+ line = line.rstrip()
266+ else:
267+ line = line.strip()
268+ if line_number == 2:
269+ # note that songs seem to start with a blank line so the title is line 2
270+ # Also we strip blanks from the title, even if ignoring indent.
271+ song_title = line.strip()
272+ # Detect the 'Reproduced from Singing the Faith Electronic Words Edition' line
273+ if line.startswith('Reproduced from Singing the Faith Electronic Words Edition'):
274+ song_number_match = re.search(r'\d+', line)
275+ if song_number_match:
276+ song_number = song_number_match.group()
277+ continue
278+ elif indent == 0:
279+ # If the indent is 0 and it contains '(c)' then it is a Copyright line
280+ if '(c)' in line:
281+ copyright = line
282+ continue
283+ elif (line.startswith('Liturgical ') or line.startswith('From The ') or
284+ line.startswith('From Common ')):
285+ self.add_comment(line)
286+ continue
287+ # If indent is 0 it may be the author, unless it was one of the cases covered above
288+ elif len(line) > 0:
289+ # May have more than one author, separated by ' and '
290+ authors = line.split(' and ')
291+ for a in authors:
292+ self.parse_author(a)
293+ continue
294+ # If a blank line has bee replaced by *Blank* then put it back to being
295+ # a simple space since this is past stripping blanks
296+ if '*Blank*' in line:
297+ line = ' '
298+ if line == '':
299+ if current_verse != '':
300+ self.add_verse(current_verse, current_verse_type)
301+ self.verse_order_list.append(current_verse_type + str(current_verse_number))
302+ if current_verse_type == 'c':
303+ chorus_written = True
304+ else:
305+ current_verse_number += 1
306+ current_verse = ''
307+ if chorus_written:
308+ current_verse_type = 'v'
309+ else:
310+ # If the line is indented more than or equal chorus_indent then assume it is a chorus
311+ # If the indent has just changed then start a new verse just like hitting a blank line
312+ if not self.hint_ignore_indent and ((indent >= chorus_indent) and (old_indent < indent)):
313+ if current_verse != '':
314+ self.add_verse(current_verse, current_verse_type)
315+ self.verse_order_list.append(current_verse_type + str(current_verse_number))
316+ if current_verse_type == 'v':
317+ current_verse_number += 1
318+ current_verse = line
319+ current_verse_type = 'c'
320+ old_indent = indent
321+ chorus_written = False
322+ has_chorus = True
323+ continue
324+ if current_verse == '':
325+ current_verse += line
326+ else:
327+ current_verse += '\n' + line
328+ old_indent = indent
329+ except Exception as e:
330+ self.log_error(translate('SongsPlugin.SingingTheFaithImport', 'File {file}').format(file=file.name),
331+ translate('SongsPlugin.SingingTheFaithImport', 'Error: {error}').format(error=e))
332+ return
333+
334+ if self.hint_song_title:
335+ song_title = self.hint_song_title
336+ self.title = '{}STF{} - {title}'.format(check_flag, song_number.zfill(3), title=song_title)
337+ self.song_book_name = 'Singing The Faith'
338+ self.song_number = song_number
339+ self.ccli_number = ccli
340+ self.add_copyright(copyright)
341+ # If we have a chorus then the generated Verse order can not be used directly, but we can generate
342+ # one for two special cases - Verse followed by one chorus (to be repeated after every verse)
343+ # of Chorus, followed by verses. If hints for ManualCheck or VerseOrder are supplied ignore this
344+ if has_chorus and not self.hint_verse_order and not self.checks_needed:
345+ auto_verse_order_ok = False
346+ # Popular case V1 C2 V2 ...
347+ if self.verse_order_list: # protect against odd cases
348+ if (self.verse_order_list[0] == 'v1') and (self.verse_order_list[1] == 'c2'):
349+ new_verse_order_list = ['v1', 'c1']
350+ i = 2
351+ auto_verse_order_ok = True
352+ elif (self.verse_order_list[0] == 'c1') and (self.verse_order_list[1] == 'v1'):
353+ new_verse_order_list = ['c1', 'v1', 'c1']
354+ i = 2
355+ auto_verse_order_ok = True
356+ # if we are in a case we can deal with
357+ if auto_verse_order_ok:
358+ while i < len(self.verse_order_list):
359+ if self.verse_order_list[i].startswith('v'):
360+ new_verse_order_list.append(self.verse_order_list[i])
361+ new_verse_order_list.append('c1')
362+ else:
363+ auto_verse_order_ok = False
364+ self.add_comment('Importer detected unexpected verse order entry {}'.format(
365+ self.verse_order_list[i]))
366+ i += 1
367+ self.verse_order_list = new_verse_order_list
368+ else:
369+ if not auto_verse_order_ok:
370+ self.verse_order_list = []
371+ if self.hint_verse_order:
372+ self.verse_order_list = self.hint_verse_order.split(',')
373+ if self.hint_comments:
374+ self.add_comment(self.hint_comments)
375+ if self.hint_ccli:
376+ self.ccli_number = self.hint_ccli
377+ # Write the title last as by now we will know if we need checks
378+ if hints_available and not self.checks_needed:
379+ check_flag = ''
380+ elif not hints_available and not has_chorus:
381+ check_flag = ''
382+ elif not hints_available and has_chorus and auto_verse_order_ok:
383+ check_flag = ''
384+ self.title = '{}STF{} - {title}'.format(check_flag, song_number.zfill(3), title=song_title)
385+ if not self.finish():
386+ self.log_error(file.name)
387+
388+ def read_hints(self, file, song_number):
389+ """
390+ Read the hints used to transform a particular song into version which can be projected,
391+ or improve the transformation process beyond the standard heuristics. Not every song will
392+ have, or need, hints.
393+ """
394+ hintfound = False
395+ self.hint_verse_order = ''
396+ self.hint_line.clear()
397+ self.hint_comments = ''
398+ self.hint_song_title = ''
399+ self.hint_ignore_indent = False
400+ self.hint_ccli = ''
401+ for tl in file:
402+ if not tl.strip():
403+ return hintfound
404+ tagval = tl.split(':')
405+ tag = tagval[0].strip()
406+ val = tagval[1].strip()
407+ if tag == 'Version':
408+ self.hint_file_version = val
409+ continue
410+ if (tag == 'Hymn') and (val == song_number):
411+ self.add_comment('Using hints version {}'.format(str(self.hint_file_version)))
412+ hintfound = True
413+ # Assume, unless the hints has ManualCheck that if hinted all will be OK
414+ self.checks_needed = False
415+ for tl in file:
416+ tagval = tl.split(':')
417+ tag = tagval[0].strip()
418+ val = tagval[1].strip()
419+ if tag == 'End':
420+ return hintfound
421+ elif tag == 'CommentsLine':
422+ vals = val.split(',')
423+ for v in vals:
424+ self.hint_line[v] = 'Comment'
425+ elif tag == 'IgnoreLine':
426+ vals = val.split(',')
427+ for v in vals:
428+ self.hint_line[v] = 'Ignore'
429+ elif tag == 'AuthorLine':
430+ vals = val.split(',')
431+ for v in vals:
432+ self.hint_line[v] = 'Author'
433+ elif tag == 'AddSpaceAfterSemi':
434+ vals = val.split(',')
435+ for v in vals:
436+ self.hint_line[v] = 'AddSpaceAfterSemi'
437+ elif tag == 'AddSpaceAfterColon':
438+ vals = val.split(',')
439+ for v in vals:
440+ self.hint_line[v] = 'AddSpaceAfterColon'
441+ elif tag == 'BlankLine':
442+ vals = val.split(',')
443+ for v in vals:
444+ self.hint_line[v] = 'BlankLine'
445+ elif tag == 'VerseOrder':
446+ self.hint_verse_order = val
447+ elif tag == 'ManualCheck':
448+ self.checks_needed = True
449+ elif tag == 'IgnoreIndent':
450+ self.hint_ignore_indent = True
451+ elif tag == 'VariantVerse':
452+ vvline = val.split(' ', 1)
453+ self.hint_line[vvline[0].strip()] = 'VariantVerse {}'.format(vvline[1].strip())
454+ elif tag == 'SongTitle':
455+ self.hint_song_title = val
456+ elif tag == 'AddComment':
457+ self.hint_comments += '\n' + val
458+ elif tag == 'CCLI':
459+ self.hint_ccli = val
460+ elif tag == 'Hymn':
461+ self.log_error(file.name, 'Missing End tag in hint for Hymn: {}'.format(song_number))
462+ else:
463+ self.log_error(file.name, 'Unknown tag {} value {}'.format(tag, val))
464+ return hintfound
465
466=== added file 'tests/functional/openlp_plugins/songs/test_singingthefaithimport.py'
467--- tests/functional/openlp_plugins/songs/test_singingthefaithimport.py 1970-01-01 00:00:00 +0000
468+++ tests/functional/openlp_plugins/songs/test_singingthefaithimport.py 2019-07-19 11:11:28 +0000
469@@ -0,0 +1,48 @@
470+# -*- coding: utf-8 -*-
471+# vim: autoindent shiftwidth=4 expandtab textwidth=120 tabstop=4 softtabstop=4
472+
473+##########################################################################
474+# OpenLP - Open Source Lyrics Projection #
475+# ---------------------------------------------------------------------- #
476+# Copyright (c) 2008-2019 OpenLP Developers #
477+# ---------------------------------------------------------------------- #
478+# This program is free software: you can redistribute it and/or modify #
479+# it under the terms of the GNU General Public License as published by #
480+# the Free Software Foundation, either version 3 of the License, or #
481+# (at your option) any later version. #
482+# #
483+# This program is distributed in the hope that it will be useful, #
484+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
485+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
486+# GNU General Public License for more details. #
487+# #
488+# You should have received a copy of the GNU General Public License #
489+# along with this program. If not, see <https://www.gnu.org/licenses/>. #
490+##########################################################################
491+"""
492+This module contains tests for the SingingTheFaith song importer.
493+"""
494+from tests.helpers.songfileimport import SongImportTestHelper
495+from tests.utils.constants import RESOURCE_PATH
496+
497+
498+TEST_PATH = RESOURCE_PATH / 'songs' / 'singingthefaith'
499+
500+
501+class TestSingingTheFaithFileImport(SongImportTestHelper):
502+
503+ def __init__(self, *args, **kwargs):
504+ self.importer_class_name = 'SingingTheFaithImport'
505+ self.importer_module_name = 'singingthefaith'
506+ super(TestSingingTheFaithFileImport, self).__init__(*args, **kwargs)
507+
508+ def test_song_import(self):
509+ """
510+ Test that loading a Singing The Faith file works correctly on various files
511+ """
512+ # Single verse
513+ self.file_import([TEST_PATH / 'H1.txt'],
514+ self.load_external_result_data(TEST_PATH / 'STF001.json'))
515+ # Whole song
516+ self.file_import([TEST_PATH / 'H2.txt'],
517+ self.load_external_result_data(TEST_PATH / 'STF002.json'))
518
519=== modified file 'tests/helpers/songfileimport.py'
520--- tests/helpers/songfileimport.py 2019-05-22 06:47:00 +0000
521+++ tests/helpers/songfileimport.py 2019-07-19 11:11:28 +0000
522@@ -123,7 +123,8 @@
523 log.debug("Song copyright imported: %s" % importer.song_number)
524 log.debug("Topics imported: %s" % importer.topics)
525
526- assert importer.title == title, 'title for %s should be "%s"' % (source_file_name, title)
527+ assert importer.title == title, \
528+ 'title for %s should be "%s" and is "%s"' % (source_file_name, title, importer.title)
529 for author in author_calls:
530 if isinstance(author, str):
531 self.mocked_add_author.assert_any_call(author)
532
533=== added directory 'tests/resources/songs/singingthefaith'
534=== added file 'tests/resources/songs/singingthefaith/H1.txt'
535--- tests/resources/songs/singingthefaith/H1.txt 1970-01-01 00:00:00 +0000
536+++ tests/resources/songs/singingthefaith/H1.txt 2019-07-19 11:11:28 +0000
537@@ -0,0 +1,9 @@
538+
539+1 Amazing Grace! how sweet the sound!
540+ That saved a wretch like me!
541+ I once was lost, but now am found;
542+ Was blind, but now I see.
543+
544+John Newton (d. 1807)
545+
546+Reproduced from Singing the Faith Electronic Words Edition, number 1 - or not as this is a hand made test file
547
548=== added file 'tests/resources/songs/singingthefaith/H2.txt'
549--- tests/resources/songs/singingthefaith/H2.txt 1970-01-01 00:00:00 +0000
550+++ tests/resources/songs/singingthefaith/H2.txt 2019-07-19 11:11:28 +0000
551@@ -0,0 +1,30 @@
552+
553+1 Amazing Grace! how sweet the sound!
554+ That saved a wretch like me!
555+ I once was lost, but now am found;
556+ Was blind, but now I see.
557+
558+2 'Twas grace that taught my heart to fear,
559+ And grace my fears relieved.
560+ How precious did that grace appear,
561+ The hour I first believed.
562+
563+3 The Lord has promised good to me,
564+ His Word my hope secures.
565+ He will my shield and portion be
566+ As long as life endures.
567+
568+4 Thro' many dangers, toils and snares
569+ I have already come.
570+ 'Tis grace that brought me safe thus far,
571+ And grace will lead me home.
572+
573+5 When we've been there ten thousand years,
574+ Bright shining as the sun,
575+ We've no less days to sing God's praise,
576+ Than when we first begun.
577+
578+
579+John Newton (d. 1807)
580+
581+Reproduced from Singing the Faith Electronic Words Edition, number 2 - or not as this is a hand made test file
582
583=== added file 'tests/resources/songs/singingthefaith/STF001.json'
584--- tests/resources/songs/singingthefaith/STF001.json 1970-01-01 00:00:00 +0000
585+++ tests/resources/songs/singingthefaith/STF001.json 2019-07-19 11:11:28 +0000
586@@ -0,0 +1,13 @@
587+{
588+ "title": "STF001 - Amazing Grace! how sweet the sound!",
589+ "authors": [
590+ "John Newton (d. 1807)"
591+ ],
592+ "verse_order_list": ["v1"],
593+ "verses": [
594+ [
595+ "Amazing Grace! how sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.",
596+ "v"
597+ ]
598+ ]
599+}
600
601=== added file 'tests/resources/songs/singingthefaith/STF002.json'
602--- tests/resources/songs/singingthefaith/STF002.json 1970-01-01 00:00:00 +0000
603+++ tests/resources/songs/singingthefaith/STF002.json 2019-07-19 11:11:28 +0000
604@@ -0,0 +1,29 @@
605+{
606+ "title": "STF002 - Amazing Grace! how sweet the sound!",
607+ "authors": [
608+ "John Newton (d. 1807)"
609+ ],
610+ "verse_order_list": ["v1", "v2", "v3", "v4", "v5"],
611+ "verses": [
612+ [
613+ "Amazing Grace! how sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.",
614+ "v"
615+ ],
616+ [
617+ "'Twas grace that taught my heart to fear,\nAnd grace my fears relieved.\nHow precious did that grace appear,\nThe hour I first believed.",
618+ "v"
619+ ],
620+ [
621+ "The Lord has promised good to me,\nHis Word my hope secures.\nHe will my shield and portion be\nAs long as life endures.",
622+ "v"
623+ ],
624+ [
625+ "Thro' many dangers, toils and snares\nI have already come.\n'Tis grace that brought me safe thus far,\nAnd grace will lead me home.",
626+ "v"
627+ ],
628+ [
629+ "When we've been there ten thousand years,\nBright shining as the sun,\nWe've no less days to sing God's praise,\nThan when we first begun.",
630+ "v"
631+ ]
632+ ]
633+}