Merge lp:~john+ubuntu-g/openlp/singingthefaith into lp:openlp

Proposed by John Lines
Status: Merged
Approved by: Raoul Snyman
Approved revision: 2911
Merged at revision: 2901
Proposed branch: lp:~john+ubuntu-g/openlp/singingthefaith
Merge into: lp:openlp
Diff against target: 1483 lines (+1360/-12)
14 files modified
openlp/plugins/songs/lib/importer.py (+23/-10)
openlp/plugins/songs/lib/importers/singingthefaith-hints.tag (+666/-0)
openlp/plugins/songs/lib/importers/singingthefaith.py (+437/-0)
tests/functional/openlp_plugins/songs/test_singingthefaithimport.py (+63/-0)
tests/helpers/songfileimport.py (+4/-2)
tests/resources/songs/singingthefaith/H901.txt (+9/-0)
tests/resources/songs/singingthefaith/H902.txt (+30/-0)
tests/resources/songs/singingthefaith/STF901.json (+13/-0)
tests/resources/songs/singingthefaith/STF902.json (+29/-0)
tests/resources/songs/singingthefaith/hints/H1.txt (+9/-0)
tests/resources/songs/singingthefaith/hints/H2.txt (+30/-0)
tests/resources/songs/singingthefaith/hints/STF001.json (+13/-0)
tests/resources/songs/singingthefaith/hints/STF002.json (+29/-0)
tests/resources/songs/singingthefaith/hints/singingthefaith-hints.tag (+5/-0)
To merge this branch: bzr merge lp:~john+ubuntu-g/openlp/singingthefaith
Reviewer Review Type Date Requested Status
Raoul Snyman Approve
Tomas Groth Approve
Phill Pending
Review via email: mp+372277@code.launchpad.net

This proposal supersedes a proposal from 2019-09-03.

Commit message

Initial merge of SingingTheFaithImport, including update to importer.py

Description of the change

Singing The Faith is the new Authorized Hymn book for the Methodist Church of Great Britain.
There is an electronic version of the Hymn book, for Windows only, which can export Hymns as text files.

This import module smooths the process of converting these text files into OpenLP. The input format is messy and not intended for automatic processing so the importer uses a combination of heuristics and a hints file. This version has not been tested on all the hymns in Singing The Faith, but deals with most of the, more than 600, hymns it has been tested with.

Documentation for the source format and hints file is at https://wiki.openlp.org/Development:SingingTheFaith_Format

The change includes a test module, which works for the single verse case, and for a whole song - both without use of hints, and another couple of cases using hints.

To post a comment you must log in.
Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests failed, please see https://ci.openlp.io/job/MP-02-Linux_Tests/191/ for more details

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

The test is failing because the "add_verse" method is not called, or not called with that exact data. I've broken up the error message into 3 parts below for easier reading.

AssertionError:

add_verse('Amazing grace! How sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.', 'v1')

call not found

review: Needs Fixing
Revision history for this message
Phill (phill-ridout) wrote : Posted in a previous version of this proposal

A few in-line comments. Also you don't need to use parenthesis around the expressions in the if statements.

Revision history for this message
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

> A few in-line comments. Also you don't need to use parenthesis around the
> expressions in the if statements.

Thanks - have updated to use Path more, and have removed redudant parentheses round expressions in if statements

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Hey John, a couple things.

1. You have a ton of linting issues. I started commenting, and then I realised that it would be better to just point you to flake8. See the link at the bottom of my comment for a quick introduction to linting and flake8.

2. You have some inconsistent indentation. Indentation in Python is very important, so we take it seriously. Also, please make sure you are indenting using spaces and not tabs.

3. You are not committing your code with the e-mail address associated with Launchpad. Please can you fix that by issuing a bzr whoami "John Lines <email address hidden>"

4. Once you've made all your changes, and you're ready for another review, you need to resubmit your merge proposal. Do this by clicking the "Resubmit" link in the top right hand corner of the page.

5. If you're struggling with anything, pop into our IRC channel, there's usually someone around who is happy to help.

https://medium.com/python-pandemonium/what-is-flake8-and-why-we-should-use-it-b89bd78073f2

review: Needs Fixing
Revision history for this message
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

1. Thanks for the info on flake8 - is now flake8 clean - you are quite right - it is better for me to learn about the tools.
2. Think indentation is fixed - flake8 was handy as well
3. Email address now set in my bzr config
4. Can you have another look at the change now.

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linting failed, please see https://ci.openlp.io/job/MP-03-Linting/132/ for more details

Revision history for this message
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

Fix lint tests outside main importer code

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linting passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

macOS tests passed!

Revision history for this message
Phill (phill-ridout) wrote : Posted in a previous version of this proposal

Please change your string formatting to use the 'new' style with the format function. ( https://pyformat.info/ ) also string formatting is preferred over concatenation (i.e, "part1" + var + "part2")

single quotes for strings, not double quotes

do_import_file is very long can this be split in to smaller methods?

review: Needs Fixing
Revision history for this message
Tomas Groth (tomasgroth) wrote : Posted in a previous version of this proposal

Generally it looks good to me, though I haven't tested. But a few things to fix.

review: Needs Fixing
Revision history for this message
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

Using new string formatting, constructor for class.

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linting passed!

Revision history for this message
Phill (phill-ridout) wrote : Posted in a previous version of this proposal

Just a few more inline comments.

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Hey John, if you're ready for us to take another look at this, make sure to resubmit it.

Revision history for this message
Tomas Groth (tomasgroth) wrote : Posted in a previous version of this proposal

Is there any reason the hint file isn't included? If it is not bundled with OpenLP, then where is the user supposed to get it from?

Revision history for this message
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

The use of STFnnn - in the title is now controlled by a hint. Also other code tidying.

Revision history for this message
Tomas Groth (tomasgroth) wrote : Posted in a previous version of this proposal

Hi John,

I still don't get why the hint file is not included...
If you already made the file, why should others have to do it? I assume that the "Singing The Faith" files that can be imported are the same for all potential users?

review: Needs Information
Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

JavaScript tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests failed, please see https://ci.openlp.io/job/MP-03-Linux-Tests/234/ for more details

Revision history for this message
Tomas Groth (tomasgroth) wrote : Posted in a previous version of this proposal

The tests are failing, see https://ci.openlp.io/job/MP-03-Linux-Tests/234/console
Also see the inline comment.

review: Needs Fixing
Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

JavaScript tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

macOS tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linting failed, please see https://ci.openlp.io/job/MP-05-Linting/166/ for more details

Revision history for this message
John Lines (john+ubuntu-g) wrote : Posted in a previous version of this proposal

> Linting failed, please see https://ci.openlp.io/job/MP-05-Linting/166/ for
> more details

now fixed - blank lines at end of test code

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

JavaScript tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linux tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

macOS tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote : Posted in a previous version of this proposal

Linting passed!

Revision history for this message
Tomas Groth (tomasgroth) wrote : Posted in a previous version of this proposal

Hi John, sorry for mentioning this before, but the hint file should be placed in the folder openlp/plugins/songs/lib/importers/
If the user does not supply a hint file you can then load it from this path:
AppLocation.get_directory(AppLocation.PluginsDir) / 'songs' / 'lib' / 'importers' / 'singingthefaith-hints.tag'
As you can see I would recommend renaming it to 'singingthefaith-hints.tag' or something similar to make it easy to see what it is used for.
And please set 'SongbookNumberInTitle' in the default hint file to False.

review: Needs Fixing
Revision history for this message
Raoul Snyman (raoul-snyman) wrote :

JavaScript tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote :

Linux tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote :

macOS tests passed!

Revision history for this message
Raoul Snyman (raoul-snyman) wrote :

Linting passed!

Revision history for this message
Tomas Groth (tomasgroth) wrote :

Looks good.

review: Approve
Revision history for this message
Raoul Snyman (raoul-snyman) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'openlp/plugins/songs/lib/importer.py'
2--- openlp/plugins/songs/lib/importer.py 2019-04-13 13:00:22 +0000
3+++ openlp/plugins/songs/lib/importer.py 2019-09-04 16:19:42 +0000
4@@ -42,6 +42,7 @@
5 from .importers.powersong import PowerSongImport
6 from .importers.presentationmanager import PresentationManagerImport
7 from .importers.propresenter import ProPresenterImport
8+from .importers.singingthefaith import SingingTheFaithImport
9 from .importers.songbeamer import SongBeamerImport
10 from .importers.songpro import SongProImport
11 from .importers.songshowplus import SongShowPlusImport
12@@ -173,16 +174,17 @@
13 PowerSong = 16
14 PresentationManager = 17
15 ProPresenter = 18
16- SongBeamer = 19
17- SongPro = 20
18- SongShowPlus = 21
19- SongsOfFellowship = 22
20- SundayPlus = 23
21- VideoPsalm = 24
22- WordsOfWorship = 25
23- WorshipAssistant = 26
24- WorshipCenterPro = 27
25- ZionWorx = 28
26+ SingingTheFaith = 19
27+ SongBeamer = 20
28+ SongPro = 21
29+ SongShowPlus = 22
30+ SongsOfFellowship = 23
31+ SundayPlus = 24
32+ VideoPsalm = 25
33+ WordsOfWorship = 26
34+ WorshipAssistant = 27
35+ WorshipCenterPro = 28
36+ ZionWorx = 29
37
38 # Set optional attribute defaults
39 __defaults__ = {
40@@ -343,6 +345,16 @@
41 'filter': '{text} (*.pro4 *.pro5 *.pro6)'.format(text=translate('SongsPlugin.ImportWizardForm',
42 'ProPresenter Song Files'))
43 },
44+ SingingTheFaith: {
45+ 'class': SingingTheFaithImport,
46+ 'name': 'SingingTheFaith',
47+ 'prefix': 'singingTheFaith',
48+ 'filter': '{text} (*.txt)'.format(text=translate('SongsPlugin.ImportWizardForm',
49+ 'Singing The Faith Exported Files')),
50+ 'descriptionText': translate('SongsPlugin.ImportWizardForm',
51+ 'First use Singing The Faith Electonic edition to export '
52+ 'the song(s) in Text format.')
53+ },
54 SongBeamer: {
55 'class': SongBeamerImport,
56 'name': 'SongBeamer',
57@@ -462,6 +474,7 @@
58 SongFormat.PowerSong,
59 SongFormat.PresentationManager,
60 SongFormat.ProPresenter,
61+ SongFormat.SingingTheFaith,
62 SongFormat.SongBeamer,
63 SongFormat.SongPro,
64 SongFormat.SongShowPlus,
65
66=== added file 'openlp/plugins/songs/lib/importers/singingthefaith-hints.tag'
67--- openlp/plugins/songs/lib/importers/singingthefaith-hints.tag 1970-01-01 00:00:00 +0000
68+++ openlp/plugins/songs/lib/importers/singingthefaith-hints.tag 2019-09-04 16:19:42 +0000
69@@ -0,0 +1,666 @@
70+Tag-STFHints-version: 1.0
71+Version: 2
72+SongbookNumberInTitle: False
73+End:
74+Hymn: 2
75+VerseOrder: V1,C1,V2,C1,V3,C1
76+End:
77+Hymn: 8
78+AddSpaceAfterColon: 2,11,20,33
79+End:
80+Hymn: 10
81+CommentsLine: 17
82+End:
83+Hymn: 11
84+CommentsLine: 24
85+End:
86+Hymn: 15
87+VerseOrder: V1,C1,V2,C1,C2,C1
88+End:
89+Hymn: 18
90+CommentsLine: 16
91+End:
92+Hymn: 19
93+CommentsLine: 8
94+End:
95+Hymn: 22
96+CommentsLine: 20
97+End:
98+Hymn: 24
99+IgnoreLine: 13
100+VerseOrder: V1,V2,V1
101+End:
102+Hymn: 26
103+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C1
104+End:
105+Hymn: 27
106+AddComment: Verse 1 is original Shona
107+SongTitle: Jesu, tawa pano
108+IgnoreLine: 2
109+CommentsLine: 31
110+End:
111+Hymn: 28
112+CommentsLine: 41
113+End:
114+Hymn: 29
115+CommentsLine: 18
116+End:
117+Hymn: 30
118+CommentsLine: 26
119+End:
120+Hymn: 35
121+VerseOrder: V1,C1,V2,C1,V3,C1
122+End:
123+Hymn: 37
124+IgnoreLine: 42
125+VerseOrder: V1,V2,C1,V3,V4,C1
126+End:
127+Hymn: 38
128+ManualCheck: Yes
129+AddComment: Make all and cantor words Bold tagged for readability
130+SongTitle: Wa wa wa Emimimo
131+End:
132+Hymn: 40
133+VariantVerse: 15 1 Blessed be the name of the Lord/Glory to the name of the Lord|blessed be the name/glory to the name
134+VariantVerse: 17 1 Blessed be the name of the Lord/Holy is the name of the Lord|blessed be the name/holy is the name
135+VerseOrder: V1,C1,V2,C1,V3,C1
136+End:
137+Hymn: 41
138+IgnoreIndent: Yes
139+IgnoreLine: 35,42
140+VerseOrder: V1,V2,V3,V4,V2,V3,V5,V3
141+End:
142+Hymn: 43
143+IgnoreIndent: Yes
144+CommentsLine: 40
145+End:
146+Hymn: 45
147+IgnoreIndent: Yes
148+CommentsLine: 104
149+End:
150+Hymn: 46
151+VerseOrder: V1,C1,V2,C1,C2,C1
152+End:
153+Hymn: 48
154+VerseOrder: V1,C1,V2,C2
155+End:
156+Hymn: 51
157+VerseOrder: V1,C1,V2,C1,V3,C1
158+End:
159+Hymn: 55
160+AddSpaceAfterSemi: 15
161+End:
162+Hymn: 60
163+CommentsLine: 22
164+End:
165+Hymn: 61
166+VerseOrder: C1,V1,C1
167+End:
168+Hymn: 64
169+IgnoreLine: 23,25
170+VerseOrder: V1,C1,C2,V2,C1,C2,C3
171+End:
172+Hymn: 65
173+VerseOrder: V1,C1,V2,C2,V3,C1,V4,C1
174+End:
175+Hymn: 68
176+IgnoreLine: 15,31
177+VerseOrder: C1,V1,C1,V2,C2,C1
178+End:
179+Hymn: 71
180+IgnoreLine: 23
181+VerseOrder: V1,C1,V2,C1,V3
182+End:
183+Hymn: 74
184+IgnoreIndent: Yes
185+End:
186+Hymn: 77
187+IgnoreLine: 32
188+CommentsLine: 37
189+VerseOrder: V1,V2,C1,V3,C1
190+End:
191+Hymn: 78
192+VerseOrder: V1,V2,V1,V2,V3
193+End:
194+Hymn: 82
195+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1
196+End:
197+Hymn: 84
198+IgnoreIndent: Yes
199+End:
200+Hymn: 86
201+CommentsLine: 86
202+End:
203+Hymn: 89
204+VerseOrder: V1,V2,C1
205+End:
206+Hymn: 92
207+IgnoreIndent: Yes
208+IgnoreLine: 2,18,48
209+SongTitle: Think of a world without any flowers
210+End:
211+Hymn: 93
212+AddSpaceAfterSemi: 9,10,11,12
213+IgnoreLine: 21,30,31,32,33
214+VerseOrder: V1,C1,V2,C1,V3,C1
215+End:
216+Hymn: 94
217+AuthorLine: 24
218+VerseOrder: V1,C1,V2,C1,V3,C1
219+End:
220+Hymn: 95
221+AddSpaceAfterSemi: 2
222+End:
223+Hymn: 98
224+IgnoreLine: 17,19,24
225+VerseOrder: V1,C1,V2,C1,C2,C1
226+AddComment: C2 is an optional Bridge
227+End:
228+Hymn: 100
229+VerseOrder: C1,V1,C1,V2,C1,V3,C1,V4,C1
230+End:
231+Hymn: 102
232+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C1
233+End:
234+Hymn: 103
235+AddSpaceAfterColon: 2,3,11,20
236+End:
237+Hymn: 105
238+VerseOrder: C1,V1,C1,V2,C1
239+End:
240+Hymn: 118
241+IgnoreLine: 15,21
242+CommentsLine: 28
243+VerseOrder: V1,C1,C1,V2,C1,V3,C1,C1,C2
244+End:
245+Hymn: 123
246+CommentsLine: 40
247+End:
248+Hymn: 140
249+IgnoreLine: 14
250+VerseOrder: V1,C1,V2,C1
251+End:
252+Hymn: 141
253+AddSpaceAfterSemi: 2
254+End:
255+Hymn: 145
256+ManualCheck: Yes
257+AddComment: Make cantor and all bold, and add to all verses
258+SongTitle: Night has fallen
259+End:
260+Hymn: 147
261+AddSpaceAfterSemi: 22
262+End:
263+Hymn: 165
264+IgnoreLine: 2,11,20,29,38
265+CommentsLine: 50
266+SongTitle: Advent candles tell their story
267+End:
268+Hymn: 166
269+CommentsLine: 40
270+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C2
271+End:
272+Hymn: 168
273+CommentsLine: 30
274+End:
275+Hymn: 170
276+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C2
277+End:
278+Hymn: 173
279+VerseOrder: V1,C1,V2,C1,V3,C2
280+End:
281+Hymn: 174
282+CommentsLine: 41
283+End:
284+Hymn: 175
285+IgnoreLine: 22,29
286+VerseOrder: V1,C1,V2,C1,C2,C1
287+End:
288+Hymn: 176
289+CommentsLine: 26
290+ManualCheck: Yes
291+End:
292+Hymn: 178
293+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C2
294+End:
295+Hymn: 186
296+IgnoreIndent: Yes
297+CommentsLine: 24
298+End:
299+Hymn: 194
300+IgnoreIndent: Yes
301+End:
302+Hymn: 200
303+AddSpaceAfterColon: 2
304+CommentsLine: 24
305+End:
306+Hymn: 209
307+IgnoreLine: 11,17,23
308+VerseOrder: V1,C1,C1,V2,C1,V3,C1
309+End:
310+Hymn: 212
311+CommentsLine: 46
312+End:
313+Hymn: 220
314+AddSpaceAfterColon: 30
315+End:
316+Hymn: 227
317+VerseOrder: V1,V2,V3,V4,V1
318+End:
319+Hymn: 228
320+CommentsLine: 49
321+End:
322+Hymn: 234
323+CommentsLine: 7
324+End:
325+Hymn: 235
326+IgnoreIndent: Yes
327+End:
328+Hymn: 240
329+CommentsLine: 24
330+End:
331+Hymn: 241
332+IgnoreLine: 39
333+VerseOrder: V1,C1,V2,C2,C3,C2
334+AddComment: is the final chorus a repeat of C1 or C2 ?
335+ManualCheck: Yes
336+End:
337+Hymn: 246
338+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1
339+End:
340+Hymn: 247
341+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C1
342+End:
343+Hymn: 248
344+AddSpaceAfterSemi: 13
345+End:
346+Hymn: 249
347+VerseOrder: C1,V1,C1,V2,C1,V3,C1,V4,C1
348+End:
349+Hymn: 252
350+AddSpaceAfterColon: 1,6,9,13,16,20,23,27,30,34
351+End:
352+Hymn: 254
353+CommentsLine: 22
354+End:
355+Hymn: 256
356+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C1,V6,C2
357+End:
358+Hymn: 258
359+IgnoreLine: 14
360+VerseOrder: C1,V1,C1
361+End:
362+Hymn: 261
363+AddSpaceAfterSemi: 4
364+End:
365+Hymn: 267
366+CommentsLine: 33
367+End:
368+Hymn: 274
369+IgnoreLine: 21,30
370+VerseOrder: V1,C1,V2,C1,C2,C1
371+End:
372+Hymn: 279
373+IgnoreLine: 35
374+VerseOrder: V1,V2,C1,V3,V4,C1
375+End:
376+Hymn: 285
377+CommentsLine: 25
378+End:
379+Hymn: 298
380+IgnoreIndent: Yes
381+AddSpaceAfterSemi: 8
382+End:
383+Hymn: 299
384+IgnoreLine: 26
385+CommentsLine: 32
386+VerseOrder: V1,C1,V2,V3,V4,C1,C2
387+End:
388+Hymn: 300
389+AddSpaceAfterSemi: 8,9,13,17,18
390+AddSpaceAfterColon: 7,15,20,25,30
391+End:
392+Hymn: 302
393+VerseOrder: C1,V1,C1,V2,C1,V3,C1,V4,C1,V5,C1
394+End:
395+Hymn: 316
396+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C1,V6,C1,V7,C2
397+End:
398+Hymn: 321
399+CommentsLine: 15
400+End:
401+Hymn: 323
402+AddSpaceAfterSemi: 11
403+End:
404+Hymn: 331
405+VerseOrder: V1,C1,V2,C2
406+End:
407+Hymn: 335
408+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C2
409+End:
410+Hymn: 343
411+IgnoreLine: 24
412+VerseOrder: V1,V2,C1,V3,C1
413+End:
414+Hymn: 349
415+AddSpaceAfterSemi: 20,24
416+IgnoreLine: 17
417+AddComment: The refrain is optional
418+VerseOrder: V1,V2,V3,C1,V4,V3
419+End:
420+Hymn: 351
421+CCLI: 3350395
422+End:
423+Hymn: 353
424+AddSpaceAfterSemi: 15
425+End:
426+Hymn: 364
427+AddSpaceAfterSemi: 22
428+End:
429+Hymn: 367
430+IgnoreLine: 28
431+VerseOrder: V1,C1,V2,C1
432+End:
433+Hymn: 373
434+CommentsLine: 26
435+End:
436+Hymn: 374
437+ManualCheck: Yes
438+CommentsLine: 24
439+End:
440+Hymn: 377
441+VerseOrder: V1,C1,V2,C1,V3,C1
442+End:
443+Hymn: 380
444+VerseOrder: V1,C1,V2,C1
445+End:
446+Hymn: 386
447+BlankLine: 6,16
448+IgnoreLine: 19,26
449+VerseOrder: V1,C1,V2,C1,C2,C1
450+End:
451+Hymn: 389
452+AuthorLine: 21
453+End:
454+Hymn: 401
455+AddSpaceAfterSemi: 27
456+End:
457+Hymn: 403
458+AddSpaceAfterColon: 2
459+End:
460+Hymn: 404
461+SongTitle: Go tell everyone
462+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1
463+End:
464+Hymn: 405
465+VerseOrder: V1,C1,V2,C1,V3,C1
466+End:
467+Hymn: 406
468+IgnoreLine: 18,27
469+End:
470+Hymn: 407
471+IgnoreLine: 31
472+VerseOrder: V1,V2,C1,V3,C1
473+End:
474+Hymn: 408
475+IgnoreIndent: Yes
476+CommentsLine: 25
477+End:
478+Hymn: 410
479+AddSpaceAfterSemi: 15,16
480+End:
481+Hymn: 419
482+CommentsLine: 25
483+End:
484+Hymn: 420
485+IgnoreIndent: Yes
486+End:
487+Hymn: 421
488+IgnoreIndent: Yes
489+CommentsLine: 55
490+End:
491+Hymn: 424
492+IgnoreLine: 16
493+VerseOrder: V1,C1,V2,C1
494+End:
495+Hymn: 428
496+IgnoreLine: 25
497+VerseOrder: V1,V2,C1,V3,C1
498+End:
499+Hymn: 432
500+AddSpaceAfterSemi: 8
501+End:
502+Hymn: 433
503+CommentsLine: 44
504+End:
505+Hymn: 447
506+IgnoreLine: 19,21
507+VerseOrder: V1,V2,C1,V3,C1,V1
508+End:
509+Hymn: 451
510+IgnoreLine: 14,16
511+VerseOrder: C1,V1,C1,V1,V2
512+End:
513+Hymn: 454
514+CommentsLine: 2,42
515+SongTitle: Where shall my wondering soul begin
516+End:
517+Hymn: 458
518+CommentsLine: 2
519+SongTitle: Away with our fears The glad morning appears
520+End:
521+Hymn: 469
522+ManualCheck: Yes
523+AddComment: Need a VariantChorus, Chorus2 and Chorus3 are Variants of Chorus1
524+VerseOrder: V1,C1,V2,C2,V3,C2,V4,C3
525+End:
526+Hymn: 470
527+IgnoreIndent: Yes
528+End:
529+Hymn: 476
530+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C1,V5,C1
531+End:
532+Hymn: 477
533+VerseOrder: C1,V1,C1,V2,C1
534+End:
535+Hymn: 480
536+CommentsLine: 28
537+End:
538+Hymn: 483
539+AddComment: Verses 4 and 5 are the
540+CommentsLine: 29
541+End:
542+Hymn: 488
543+IgnoreLine: 25
544+VerseOrder: V1,V2,C1,V3,C1
545+End:
546+Hymn: 492
547+AddSpaceAfterSemi: 9,10,15
548+End:
549+Hymn: 499
550+CommentsLine: 37
551+End:
552+Hymn: 509
553+IgnoreIndent: Yes
554+CommentsLine: 40
555+End:
556+Hymn: 517
557+VerseOrder: V1,C1,V2,C1,V3,C1,V4,C2
558+End:
559+Hymn: 528
560+CommentsLine: 32
561+End:
562+Hymn: 541
563+IgnoreLine: 17,18,25
564+VerseOrder: V1,V2,C1,V1,V2,V3,C1
565+End:
566+Hymn: 545
567+CommentsLine: 28
568+End:
569+Hymn: 548
570+AuthorLine: 22
571+End:
572+Hymn: 554
573+IgnoreLine: 15,21
574+VerseOrder: V1,C1,V2,C1,V3,C1,V4
575+End:
576+Hymn: 555
577+IgnoreLine: 15
578+VerseOrder: V1,V2,C1,V1
579+End:
580+Hymn: 559
581+IgnoreIndent: Yes
582+End:
583+Hymn: 561
584+CommentsLine: 45
585+End:
586+Hymn: 562
587+CommentsLine: 38
588+End:
589+Hymn: 565
590+IgnoreLine: 21
591+VerseOrder: V1,V2,V1
592+End:
593+Hymn: 566
594+AddSpaceAfterSemi: 27
595+End:
596+Hymn: 567
597+ManualCheck: Yes
598+AddComment: Check with musician what should be on the screen
599+End:
600+Hymn: 570
601+VerseOrder: V1,C1,V2,C1,V3,C1
602+End:
603+Hymn: 575
604+CommentsLine: 30
605+End:
606+Hymn: 578
607+IgnoreLine: 11,18,25
608+VerseOrder: C1,V1,C1,V2,C1,V3,C1
609+End:
610+Hymn: 586
611+IgnoreLine: 22
612+VerseOrder: V1,V2,C1,V3,C1
613+End:
614+Hymn: 587
615+VerseOrder: V1,C1,V2,C2,V3
616+End:
617+Hymn: 582
618+CommentsLine: 25
619+End:
620+Hymn: 594
621+CommentsLine: 41
622+End:
623+Hymn: 601
624+CommentsLine: 16
625+End:
626+Hymn: 603
627+ManualCheck: Yes
628+AddComment: Update Verse 4 for names of the couple
629+CommentsLine: 22
630+IgnoreIndent: Yes
631+End:
632+Hymn: 609
633+CommentsLine: 37
634+End:
635+Hymn: 610
636+CommentsLine: 29
637+End:
638+Hymn: 626
639+IgnoreLine: 29,31,44
640+VerseOrder: V1,C1,C2,V2,C1,C2,C3,C2
641+End:
642+Hymn: 631
643+CommentsLine: 15
644+End:
645+Hymn: 627
646+IgnoreLine: 25,36
647+VerseOrder: V1,C1,V2,C1,C2,C1
648+End:
649+Hymn: 632
650+IgnoreLine: 17,24
651+VerseOrder: V1,C1,V2,C1,C2,C1
652+End:
653+Hymn: 635
654+IgnoreLine: 23,25
655+VerseOrder: V1,C1,C2,V2,C1,C2
656+End:
657+Hymn: 637
658+CommentsLine: 41
659+End:
660+Hymn: 638
661+CommentsLine: 34
662+End:
663+Hymn: 640
664+CommentsLine: 19,20
665+End:
666+Hymn: 654
667+IgnoreIndent: Yes
668+End:
669+Hymn: 657
670+IgnoreLine: 20,21,23
671+VerseOrder: V1,C1,C2,V2,C1,C2,C3
672+End:
673+Hymn: 662
674+AddSpaceAfterSemi: 2,12
675+End:
676+Hymn: 670
677+IgnoreLine: 27
678+VerseOrder: V1,C1,V2,C1
679+End:
680+Hymn: 677
681+CommentsLine: 31
682+End:
683+Hymn: 681
684+IgnoreIndent: Yes
685+End:
686+Hymn: 684
687+IgnoreIndent: Yes
688+End:
689+Hymn: 693
690+IgnoreLine: 46,57
691+VerseOrder: V1,V2,C1,V3,V4,C1,V5,C1
692+End:
693+Hymn: 697
694+IgnoreIndent: Yes
695+CommentsLine: 21
696+End:
697+Hymn: 699
698+IgnoreLine: 23,32
699+VerseOrder: V1,C1,V2,C1,C1,C2,C1
700+End:
701+Hymn: 700
702+IgnoreIndent: Yes
703+End:
704+Hymn: 707
705+IgnoreLine: 15
706+VerseOrder: V1,C1,V2,C1,V3
707+End:
708+Hymn: 729
709+IgnoreIndent: Yes
710+End:
711+Hymn: 741
712+CommentsLine: 25
713+End:
714+Hymn: 753
715+CommentsLine: 14
716+IgnoreIndent: Yes
717+End:
718+Hymn: 754
719+IgnoreIndent: Yes
720+End:
721+Hymn: 764
722+IgnoreIndent: Yes
723+CommentsLine: 31
724+End:
725+Hymn: 783
726+BlankLine: 5
727+End:
728+Hymn: 819
729+AddSpaceAfterSemi: 32
730+End:
731+Hymn: 820
732+CommentsLine: 2
733+BoldLine: 6,7,12,13,17,18,23,24,29,30,32,33,34,35
734+SongTitle: Psalm 98 - O sing to the Lord a new song
735+End:
736
737=== added file 'openlp/plugins/songs/lib/importers/singingthefaith.py'
738--- openlp/plugins/songs/lib/importers/singingthefaith.py 1970-01-01 00:00:00 +0000
739+++ openlp/plugins/songs/lib/importers/singingthefaith.py 2019-09-04 16:19:42 +0000
740@@ -0,0 +1,437 @@
741+# -*- coding: utf-8 -*-
742+# vim: autoindent shiftwidth=4 expandtab textwidth=120 tabstop=4 softtabstop=4
743+
744+###############################################################################
745+# OpenLP - Open Source Lyrics Projection #
746+# --------------------------------------------------------------------------- #
747+# Copyright (c) 2008-2019 OpenLP Developers #
748+# --------------------------------------------------------------------------- #
749+# This program is free software; you can redistribute it and/or modify it #
750+# under the terms of the GNU General Public License as published by the Free #
751+# Software Foundation; version 3 of the License. #
752+# #
753+# This program is distributed in the hope that it will be useful, but WITHOUT #
754+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or #
755+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for #
756+# more details. #
757+# #
758+# You should have received a copy of the GNU General Public License along #
759+# with this program; if not, write to the Free Software Foundation, Inc., 59 #
760+# Temple Place, Suite 330, Boston, MA 02111-1307 USA #
761+###############################################################################
762+"""
763+The :mod:`singingthefaith` module provides the functionality for importing songs which are
764+exported from Singing The Faith - an Authorised songbook for the Methodist Church of
765+Great Britain."""
766+
767+import logging
768+import re
769+from pathlib import Path
770+
771+from openlp.core.common.i18n import translate
772+from openlp.plugins.songs.lib.importers.songimport import SongImport
773+from openlp.core.common.applocation import AppLocation
774+
775+
776+log = logging.getLogger(__name__)
777+
778+
779+class SingingTheFaithImport(SongImport):
780+ """
781+ Import songs exported from SingingTheFaith
782+ """
783+
784+ def __init__(self, manager, **kwargs):
785+ """
786+ Initialise the class.
787+ """
788+ super(SingingTheFaithImport, self).__init__(manager, **kwargs)
789+ self.hints_available = False
790+ self.checks_needed = True
791+ self.hint_line = {}
792+ self.hint_file_version = '0'
793+ self.hint_verse_order = ''
794+ self.hint_song_title = ''
795+ self.hint_comments = ''
796+ self.hint_ccli = ''
797+ self.hint_ignore_indent = False
798+ self.hint_songbook_number_in_title = False
799+
800+ def do_import(self):
801+ """
802+ Receive a single file or a list of files to import.
803+ """
804+ if not isinstance(self.import_source, list):
805+ return
806+ self.import_wizard.progress_bar.setMaximum(len(self.import_source))
807+ for file_path in self.import_source:
808+ if self.stop_import_flag:
809+ return
810+ # If this is backported to version 2.4 then do_import is called with a filename
811+ # rather than a path object if called from the development version.
812+ # Check here to minimise differences between versions.
813+ if isinstance(file_path, str):
814+ song_file = open(file_path, 'rt', encoding='cp1251')
815+ self.do_import_file(song_file)
816+ song_file.close()
817+ else:
818+ with file_path.open('rt', encoding='cp1251') as song_file:
819+ self.do_import_file(song_file)
820+
821+ def do_import_file(self, file):
822+ """
823+ Process the SingingTheFaith file - pass in a file-like object, not a file path.
824+ """
825+ hints_file_name = 'singingthefaith-hints.tag'
826+ singing_the_faith_version = 1
827+ self.set_defaults()
828+ # Setup variables
829+ line_number = 0
830+ old_indent = 0
831+ # The chorus indent is how many spaces the chorus is indented - it might be 6,
832+ # but we test for >= and I do not know how consistent to formatting of the
833+ # exported songs is.
834+ chorus_indent = 5
835+ # Initialise the song title - the format of the title finally produced can be affected
836+ # by the SongbookNumberInTitle option in the hints file
837+ song_title = 'STF000 -'
838+ song_number = '0'
839+ ccli = '0'
840+ current_verse = ''
841+ current_verse_type = 'v'
842+ current_verse_number = 1
843+ # Potentially we could try to track current chorus number to automatically handle
844+ # more than 1 chorus, currently unused.
845+ # current_chorus_number = 1
846+ has_chorus = False
847+ chorus_written = False
848+ auto_verse_order_ok = False
849+ copyright = ''
850+ # the check_flag is prepended to the title, removed if the import should be OK
851+ # all the songs which need manual editing should sort below all the OK songs
852+ check_flag = 'z'
853+
854+ self.add_comment(
855+ 'Imported with Singing The Faith Importer v{no}'.format(no=singing_the_faith_version))
856+
857+ # Get the file_song_number - so we can use it for hints
858+ filename = Path(file.name)
859+ song_number_file = filename.stem
860+ song_number_match = re.search(r'\d+', song_number_file)
861+ if song_number_match:
862+ song_number_file = song_number_match.group()
863+
864+ # See if there is a hints file in the same location as the file
865+ dir_path = filename.parent
866+ hints_file_path = dir_path / hints_file_name
867+ try:
868+ with hints_file_path.open('r') as hints_file:
869+ hints_available = self.read_hints(hints_file, song_number_file)
870+ except FileNotFoundError:
871+ # Look for the hints file in the Plugins directory
872+ hints_file_path = AppLocation.get_directory(AppLocation.PluginsDir) / 'songs' / 'lib' / \
873+ 'importers' / hints_file_name
874+ try:
875+ with hints_file_path.open('r') as hints_file:
876+ hints_available = self.read_hints(hints_file, song_number_file)
877+ except FileNotFoundError:
878+ hints_available = False
879+
880+ try:
881+ for line in file:
882+ line_number += 1
883+ # Strip out leftover formatting (\i and \b)
884+ line = line.replace('\\i', '')
885+ line = line.replace('\\b', '')
886+ if hints_available and str(line_number) in self.hint_line:
887+ hint = self.hint_line[str(line_number)]
888+ # Set to false if this hint does not replace the line
889+ line_replaced = True
890+ if hint == 'Comment':
891+ line.strip()
892+ self.add_comment(line)
893+ continue
894+ elif hint == 'Ignore':
895+ continue
896+ elif hint == 'Author':
897+ # add as a raw author - do not split
898+ line.strip()
899+ self.add_author(line)
900+ line_number += 1
901+ next(file)
902+ continue
903+ elif hint.startswith('VariantVerse'):
904+ vv, hintverse, replace = hint.split(' ', 2)
905+ this_verse = self.verses[int(hintverse) - 1]
906+ this_verse_str = this_verse[1]
907+ new_verse = this_verse_str
908+ # There might be multiple replace pairs separated by |
909+ replaces = replace.split('|')
910+ for rep in replaces:
911+ source_str, dest_str = rep.split('/')
912+ new_verse = new_verse.replace(source_str, dest_str)
913+ self.add_verse(new_verse, 'v')
914+ self.verse_order_list.append('v{}'.format(str(current_verse_number)))
915+ current_verse_number += 1
916+ line_number += 1
917+ next(file)
918+ continue
919+ elif hint == 'AddSpaceAfterSemi':
920+ line = line.replace(';', '; ')
921+ line_replaced = False
922+ # note - do not use contine here as the line should now be processed as normal.
923+ elif hint == 'AddSpaceAfterColon':
924+ line = line.replace(':', ': ')
925+ line_replaced = False
926+ elif hint == 'BlankLine':
927+ line = ' *Blank*'
928+ line_replaced = False
929+ elif hint == 'BoldLine':
930+ # processing of the hint is deferred, but pick it up as a known hint here
931+ line_replaced = False
932+ else:
933+ self.log_error(translate('SongsPlugin.SingingTheFaithImport',
934+ 'File {file})'.format(file=file.name)),
935+ translate('SongsPlugin.SingingTheFaithImport',
936+ 'Unknown hint {hint}').format(hint=hint))
937+ if line_replaced:
938+ return
939+ # STF exported lines have a leading verse number at the start of each verse.
940+ # remove them - note that we want to track the indent as that shows a chorus
941+ # so will deal with that before stripping all leading spaces.
942+ indent = 0
943+ if line.strip():
944+ # One hymn has one line which starts '* 6' at the start of a verse
945+ # Strip this out
946+ if line.startswith('* 6'):
947+ line = line.lstrip('* ')
948+ verse_num_match = re.search(r'^\d+', line)
949+ if verse_num_match:
950+ # Could extract the verse number and check it against the calculated
951+ # verse number - TODO
952+ # verse_num = verse_num_match.group()
953+ line = line.lstrip('0123456789')
954+ indent_match = re.search(r'^\s+', line)
955+ if indent_match:
956+ indent = len(indent_match.group())
957+ # Assuming we have sorted out what is verse and what is chorus, strip lines,
958+ # unless ignoreIndent
959+ if self.hint_ignore_indent:
960+ line = line.rstrip()
961+ else:
962+ line = line.strip()
963+ if line_number == 2:
964+ # note that songs seem to start with a blank line so the title is line 2
965+ # Also we strip blanks from the title, even if ignoring indent.
966+ song_title = line.strip()
967+ # Process possible line formatting hints after the verse number has been removed
968+ if hints_available and str(line_number) in self.hint_line and hint == 'BoldLine':
969+ line = '{{st}}{0}{{/st}}'.format(line)
970+ # Detect the 'Reproduced from Singing the Faith Electronic Words Edition' line
971+ if line.startswith('Reproduced from Singing the Faith Electronic Words Edition'):
972+ song_number_match = re.search(r'\d+', line)
973+ if song_number_match:
974+ song_number = song_number_match.group()
975+ continue
976+ elif indent == 0:
977+ # If the indent is 0 and it contains '(c)' then it is a Copyright line
978+ if '(c)' in line:
979+ copyright = line
980+ continue
981+ elif (line.startswith('Liturgical ') or line.startswith('From The ') or
982+ line.startswith('From Common ') or line.startswith('Based on Psalm ')):
983+ self.add_comment(line)
984+ continue
985+ # If indent is 0 it may be the author, unless it was one of the cases covered above
986+ elif len(line) > 0:
987+ # May have more than one author, separated by ' and '
988+ authors = line.split(' and ')
989+ for a in authors:
990+ self.parse_author(a)
991+ continue
992+ # If a blank line has bee replaced by *Blank* then put it back to being
993+ # a simple space since this is past stripping blanks
994+ if '*Blank*' in line:
995+ line = ' '
996+ if line == '':
997+ if current_verse != '':
998+ self.add_verse(current_verse, current_verse_type)
999+ self.verse_order_list.append(current_verse_type + str(current_verse_number))
1000+ if current_verse_type == 'c':
1001+ chorus_written = True
1002+ else:
1003+ current_verse_number += 1
1004+ current_verse = ''
1005+ if chorus_written:
1006+ current_verse_type = 'v'
1007+ else:
1008+ # If the line is indented more than or equal chorus_indent then assume it is a chorus
1009+ # If the indent has just changed then start a new verse just like hitting a blank line
1010+ if not self.hint_ignore_indent and ((indent >= chorus_indent) and (old_indent < indent)):
1011+ if current_verse != '':
1012+ self.add_verse(current_verse, current_verse_type)
1013+ self.verse_order_list.append(current_verse_type + str(current_verse_number))
1014+ if current_verse_type == 'v':
1015+ current_verse_number += 1
1016+ current_verse = line
1017+ current_verse_type = 'c'
1018+ old_indent = indent
1019+ chorus_written = False
1020+ has_chorus = True
1021+ continue
1022+ if current_verse == '':
1023+ current_verse += line
1024+ else:
1025+ current_verse += '\n' + line
1026+ old_indent = indent
1027+ except Exception as e:
1028+ self.log_error(translate('SongsPlugin.SingingTheFaithImport', 'File {file}').format(file=file.name),
1029+ translate('SongsPlugin.SingingTheFaithImport', 'Error: {error}').format(error=e))
1030+ return
1031+
1032+ if self.hint_song_title:
1033+ song_title = self.hint_song_title
1034+ self.title = '{}STF{} - {title}'.format(check_flag, song_number.zfill(3), title=song_title)
1035+ self.song_book_name = 'Singing The Faith'
1036+ self.song_number = song_number
1037+ self.ccli_number = ccli
1038+ self.add_copyright(copyright)
1039+ # If we have a chorus then the generated Verse order can not be used directly, but we can generate
1040+ # one for two special cases - Verse followed by one chorus (to be repeated after every verse)
1041+ # of Chorus, followed by verses. If hints for ManualCheck or VerseOrder are supplied ignore this
1042+ if has_chorus and not self.hint_verse_order and not self.checks_needed:
1043+ auto_verse_order_ok = False
1044+ # Popular case V1 C2 V2 ...
1045+ if self.verse_order_list: # protect against odd cases
1046+ if self.verse_order_list[0] == 'v1' and self.verse_order_list[1] == 'c2':
1047+ new_verse_order_list = ['v1', 'c1']
1048+ i = 2
1049+ auto_verse_order_ok = True
1050+ elif self.verse_order_list[0] == 'c1' and self.verse_order_list[1] == 'v1':
1051+ new_verse_order_list = ['c1', 'v1', 'c1']
1052+ i = 2
1053+ auto_verse_order_ok = True
1054+ # if we are in a case we can deal with
1055+ if auto_verse_order_ok:
1056+ while i < len(self.verse_order_list):
1057+ if self.verse_order_list[i].startswith('v'):
1058+ new_verse_order_list.append(self.verse_order_list[i])
1059+ new_verse_order_list.append('c1')
1060+ else:
1061+ auto_verse_order_ok = False
1062+ self.add_comment('Importer detected unexpected verse order entry {}'.format(
1063+ self.verse_order_list[i]))
1064+ i += 1
1065+ self.verse_order_list = new_verse_order_list
1066+ else:
1067+ if not auto_verse_order_ok:
1068+ self.verse_order_list = []
1069+ if self.hint_verse_order:
1070+ self.verse_order_list = self.hint_verse_order.split(',')
1071+ if self.hint_comments:
1072+ self.add_comment(self.hint_comments)
1073+ if self.hint_ccli:
1074+ self.ccli_number = self.hint_ccli
1075+ # Write the title last as by now we will know if we need checks
1076+ if hints_available and not self.checks_needed:
1077+ check_flag = ''
1078+ elif not hints_available and not has_chorus:
1079+ check_flag = ''
1080+ elif not hints_available and has_chorus and auto_verse_order_ok:
1081+ check_flag = ''
1082+ if self.hint_songbook_number_in_title:
1083+ self.title = '{}STF{} - {title}'.format(check_flag, song_number.zfill(3), title=song_title)
1084+ else:
1085+ self.title = '{}{title}'.format(check_flag, title=song_title)
1086+ if not self.finish():
1087+ self.log_error(file.name)
1088+
1089+ def read_hints(self, file, song_number):
1090+ """
1091+ Read the hints used to transform a particular song into version which can be projected,
1092+ or improve the transformation process beyond the standard heuristics. Not every song will
1093+ have, or need, hints.
1094+ """
1095+ hintfound = False
1096+ self.hint_verse_order = ''
1097+ self.hint_line.clear()
1098+ self.hint_comments = ''
1099+ self.hint_song_title = ''
1100+ self.hint_ignore_indent = False
1101+ self.hint_ccli = ''
1102+ for tl in file:
1103+ if not tl.strip():
1104+ return hintfound
1105+ tagval = tl.split(':')
1106+ tag = tagval[0].strip()
1107+ val = tagval[1].strip()
1108+ if tag == 'Version':
1109+ self.hint_file_version = val
1110+ continue
1111+ elif tag == 'SongbookNumberInTitle':
1112+ if val == 'False':
1113+ self.hint_songbook_number_in_title = False
1114+ else:
1115+ self.hint_songbook_number_in_title = True
1116+ continue
1117+ elif tag == 'Comment':
1118+ continue
1119+ if (tag == 'Hymn') and (val == song_number):
1120+ self.add_comment('Using hints version {}'.format(str(self.hint_file_version)))
1121+ hintfound = True
1122+ # Assume, unless the hints has ManualCheck that if hinted all will be OK
1123+ self.checks_needed = False
1124+ for tl in file:
1125+ tagval = tl.split(':')
1126+ tag = tagval[0].strip()
1127+ val = tagval[1].strip()
1128+ if tag == 'End':
1129+ return hintfound
1130+ elif tag == 'CommentsLine':
1131+ vals = val.split(',')
1132+ for v in vals:
1133+ self.hint_line[v] = 'Comment'
1134+ elif tag == 'IgnoreLine':
1135+ vals = val.split(',')
1136+ for v in vals:
1137+ self.hint_line[v] = 'Ignore'
1138+ elif tag == 'AuthorLine':
1139+ vals = val.split(',')
1140+ for v in vals:
1141+ self.hint_line[v] = 'Author'
1142+ elif tag == 'AddSpaceAfterSemi':
1143+ vals = val.split(',')
1144+ for v in vals:
1145+ self.hint_line[v] = 'AddSpaceAfterSemi'
1146+ elif tag == 'AddSpaceAfterColon':
1147+ vals = val.split(',')
1148+ for v in vals:
1149+ self.hint_line[v] = 'AddSpaceAfterColon'
1150+ elif tag == 'BlankLine':
1151+ vals = val.split(',')
1152+ for v in vals:
1153+ self.hint_line[v] = 'BlankLine'
1154+ elif tag == 'BoldLine':
1155+ vals = val.split(',')
1156+ for v in vals:
1157+ self.hint_line[v] = 'BoldLine'
1158+ elif tag == 'VerseOrder':
1159+ self.hint_verse_order = val
1160+ elif tag == 'ManualCheck':
1161+ self.checks_needed = True
1162+ elif tag == 'IgnoreIndent':
1163+ self.hint_ignore_indent = True
1164+ elif tag == 'VariantVerse':
1165+ vvline = val.split(' ', 1)
1166+ self.hint_line[vvline[0].strip()] = 'VariantVerse {}'.format(vvline[1].strip())
1167+ elif tag == 'SongTitle':
1168+ self.hint_song_title = val
1169+ elif tag == 'AddComment':
1170+ self.hint_comments += '\n' + val
1171+ elif tag == 'CCLI':
1172+ self.hint_ccli = val
1173+ elif tag == 'Hymn':
1174+ self.log_error(file.name, 'Missing End tag in hint for Hymn: {}'.format(song_number))
1175+ else:
1176+ self.log_error(file.name, 'Unknown tag {} value {}'.format(tag, val))
1177+ return hintfound
1178
1179=== added file 'tests/functional/openlp_plugins/songs/test_singingthefaithimport.py'
1180--- tests/functional/openlp_plugins/songs/test_singingthefaithimport.py 1970-01-01 00:00:00 +0000
1181+++ tests/functional/openlp_plugins/songs/test_singingthefaithimport.py 2019-09-04 16:19:42 +0000
1182@@ -0,0 +1,63 @@
1183+# -*- coding: utf-8 -*-
1184+# vim: autoindent shiftwidth=4 expandtab textwidth=120 tabstop=4 softtabstop=4
1185+
1186+##########################################################################
1187+# OpenLP - Open Source Lyrics Projection #
1188+# ---------------------------------------------------------------------- #
1189+# Copyright (c) 2008-2019 OpenLP Developers #
1190+# ---------------------------------------------------------------------- #
1191+# This program is free software: you can redistribute it and/or modify #
1192+# it under the terms of the GNU General Public License as published by #
1193+# the Free Software Foundation, either version 3 of the License, or #
1194+# (at your option) any later version. #
1195+# #
1196+# This program is distributed in the hope that it will be useful, #
1197+# but WITHOUT ANY WARRANTY; without even the implied warranty of #
1198+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
1199+# GNU General Public License for more details. #
1200+# #
1201+# You should have received a copy of the GNU General Public License #
1202+# along with this program. If not, see <https://www.gnu.org/licenses/>. #
1203+##########################################################################
1204+"""
1205+This module contains tests for the SingingTheFaith song importer.
1206+"""
1207+from tests.helpers.songfileimport import SongImportTestHelper
1208+from tests.utils.constants import RESOURCE_PATH
1209+
1210+
1211+TEST_PATH = RESOURCE_PATH / 'songs' / 'singingthefaith'
1212+
1213+
1214+class TestSingingTheFaithFileImport(SongImportTestHelper):
1215+
1216+ def __init__(self, *args, **kwargs):
1217+ self.importer_class_name = 'SingingTheFaithImport'
1218+ self.importer_module_name = 'singingthefaith'
1219+ super(TestSingingTheFaithFileImport, self).__init__(*args, **kwargs)
1220+
1221+ def test_song_import(self):
1222+ """
1223+ Test that loading a Singing The Faith file works correctly on various files
1224+ """
1225+ # Note that the previous tests without hints no longer apply as there is always a
1226+ # hints file, which contains the hints for the real Singing The Faith Songs.
1227+ # Unhinted songs here must be numbered above any real Singing The Faith Song
1228+ # Single verse
1229+ self.file_import([TEST_PATH / 'H901.txt'],
1230+ self.load_external_result_data(TEST_PATH / 'STF901.json'))
1231+ # Whole song
1232+ self.file_import([TEST_PATH / 'H902.txt'],
1233+ self.load_external_result_data(TEST_PATH / 'STF902.json'))
1234+
1235+ # Tests with hints - note that the hints directory has a hints.tag which specifies
1236+ # SongbookNumberInTitle: True
1237+ # The default is false, so the unhinted tests will not have the title, but the hinted
1238+ # song tests will need it
1239+
1240+ # Single verse
1241+ self.file_import([TEST_PATH / 'hints' / 'H1.txt'],
1242+ self.load_external_result_data(TEST_PATH / 'hints' / 'STF001.json'))
1243+ # Whole song
1244+ self.file_import([TEST_PATH / 'hints' / 'H2.txt'],
1245+ self.load_external_result_data(TEST_PATH / 'hints' / 'STF002.json'))
1246
1247=== modified file 'tests/helpers/songfileimport.py'
1248--- tests/helpers/songfileimport.py 2019-05-22 06:47:00 +0000
1249+++ tests/helpers/songfileimport.py 2019-09-04 16:19:42 +0000
1250@@ -123,7 +123,8 @@
1251 log.debug("Song copyright imported: %s" % importer.song_number)
1252 log.debug("Topics imported: %s" % importer.topics)
1253
1254- assert importer.title == title, 'title for %s should be "%s"' % (source_file_name, title)
1255+ assert importer.title == title, \
1256+ 'title for %s should be "%s" and is "%s"' % (source_file_name, title, importer.title)
1257 for author in author_calls:
1258 if isinstance(author, str):
1259 self.mocked_add_author.assert_any_call(author)
1260@@ -152,7 +153,8 @@
1261 'song_number for %s should be %s' % (source_file_name, song_number)
1262 if verse_order_list:
1263 assert importer.verse_order_list == verse_order_list, \
1264- 'verse_order_list for %s should be %s' % (source_file_name, verse_order_list)
1265+ 'verse_order_list for %s should be %s and is %s' % (source_file_name,
1266+ verse_order_list, importer.verse_order_list)
1267 self.mocked_finish.assert_called_with()
1268
1269 def _get_data(self, data, key):
1270
1271=== added directory 'tests/resources/songs/singingthefaith'
1272=== added file 'tests/resources/songs/singingthefaith/H901.txt'
1273--- tests/resources/songs/singingthefaith/H901.txt 1970-01-01 00:00:00 +0000
1274+++ tests/resources/songs/singingthefaith/H901.txt 2019-09-04 16:19:42 +0000
1275@@ -0,0 +1,9 @@
1276+
1277+1 Amazing Grace! how sweet the sound!
1278+ That saved a wretch like me!
1279+ I once was lost, but now am found;
1280+ Was blind, but now I see.
1281+
1282+John Newton (d. 1807)
1283+
1284+Reproduced from Singing the Faith Electronic Words Edition, number 901 - or not as this is a hand made test file
1285
1286=== added file 'tests/resources/songs/singingthefaith/H902.txt'
1287--- tests/resources/songs/singingthefaith/H902.txt 1970-01-01 00:00:00 +0000
1288+++ tests/resources/songs/singingthefaith/H902.txt 2019-09-04 16:19:42 +0000
1289@@ -0,0 +1,30 @@
1290+
1291+1 Amazing Grace! how sweet the sound!
1292+ That saved a wretch like me!
1293+ I once was lost, but now am found;
1294+ Was blind, but now I see.
1295+
1296+2 'Twas grace that taught my heart to fear,
1297+ And grace my fears relieved.
1298+ How precious did that grace appear,
1299+ The hour I first believed.
1300+
1301+3 The Lord has promised good to me,
1302+ His Word my hope secures.
1303+ He will my shield and portion be
1304+ As long as life endures.
1305+
1306+4 Thro' many dangers, toils and snares
1307+ I have already come.
1308+ 'Tis grace that brought me safe thus far,
1309+ And grace will lead me home.
1310+
1311+5 When we've been there ten thousand years,
1312+ Bright shining as the sun,
1313+ We've no less days to sing God's praise,
1314+ Than when we first begun.
1315+
1316+
1317+John Newton (d. 1807)
1318+
1319+Reproduced from Singing the Faith Electronic Words Edition, number 2 - or not as this is a hand made test file
1320
1321=== added file 'tests/resources/songs/singingthefaith/STF901.json'
1322--- tests/resources/songs/singingthefaith/STF901.json 1970-01-01 00:00:00 +0000
1323+++ tests/resources/songs/singingthefaith/STF901.json 2019-09-04 16:19:42 +0000
1324@@ -0,0 +1,13 @@
1325+{
1326+ "title": "Amazing Grace! how sweet the sound!",
1327+ "authors": [
1328+ "John Newton (d. 1807)"
1329+ ],
1330+ "verse_order_list": ["v1"],
1331+ "verses": [
1332+ [
1333+ "Amazing Grace! how sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.",
1334+ "v"
1335+ ]
1336+ ]
1337+}
1338
1339=== added file 'tests/resources/songs/singingthefaith/STF902.json'
1340--- tests/resources/songs/singingthefaith/STF902.json 1970-01-01 00:00:00 +0000
1341+++ tests/resources/songs/singingthefaith/STF902.json 2019-09-04 16:19:42 +0000
1342@@ -0,0 +1,29 @@
1343+{
1344+ "title": "Amazing Grace! how sweet the sound!",
1345+ "authors": [
1346+ "John Newton (d. 1807)"
1347+ ],
1348+ "verse_order_list": ["v1", "v2", "v3", "v4", "v5"],
1349+ "verses": [
1350+ [
1351+ "Amazing Grace! how sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.",
1352+ "v"
1353+ ],
1354+ [
1355+ "'Twas grace that taught my heart to fear,\nAnd grace my fears relieved.\nHow precious did that grace appear,\nThe hour I first believed.",
1356+ "v"
1357+ ],
1358+ [
1359+ "The Lord has promised good to me,\nHis Word my hope secures.\nHe will my shield and portion be\nAs long as life endures.",
1360+ "v"
1361+ ],
1362+ [
1363+ "Thro' many dangers, toils and snares\nI have already come.\n'Tis grace that brought me safe thus far,\nAnd grace will lead me home.",
1364+ "v"
1365+ ],
1366+ [
1367+ "When we've been there ten thousand years,\nBright shining as the sun,\nWe've no less days to sing God's praise,\nThan when we first begun.",
1368+ "v"
1369+ ]
1370+ ]
1371+}
1372
1373=== added directory 'tests/resources/songs/singingthefaith/hints'
1374=== added file 'tests/resources/songs/singingthefaith/hints/H1.txt'
1375--- tests/resources/songs/singingthefaith/hints/H1.txt 1970-01-01 00:00:00 +0000
1376+++ tests/resources/songs/singingthefaith/hints/H1.txt 2019-09-04 16:19:42 +0000
1377@@ -0,0 +1,9 @@
1378+
1379+1 Amazing Grace! how sweet the sound!
1380+ That saved a wretch like me!
1381+ I once was lost, but now am found;
1382+ Was blind, but now I see.
1383+
1384+John Newton (d. 1807)
1385+
1386+Reproduced from Singing the Faith Electronic Words Edition, number 1 - or not as this is a hand made test file
1387
1388=== added file 'tests/resources/songs/singingthefaith/hints/H2.txt'
1389--- tests/resources/songs/singingthefaith/hints/H2.txt 1970-01-01 00:00:00 +0000
1390+++ tests/resources/songs/singingthefaith/hints/H2.txt 2019-09-04 16:19:42 +0000
1391@@ -0,0 +1,30 @@
1392+
1393+1 Amazing Grace! how sweet the sound!
1394+ That saved a wretch like me!
1395+ I once was lost, but now am found;
1396+ Was blind, but now I see.
1397+
1398+2 'Twas grace that taught my heart to fear,
1399+ And grace my fears relieved.
1400+ How precious did that grace appear,
1401+ The hour I first believed.
1402+
1403+3 The Lord has promised good to me,
1404+ His Word my hope secures.
1405+ He will my shield and portion be
1406+ As long as life endures.
1407+
1408+4 Thro' many dangers, toils and snares
1409+ I have already come.
1410+ 'Tis grace that brought me safe thus far,
1411+ And grace will lead me home.
1412+
1413+5 When we've been there ten thousand years,
1414+ Bright shining as the sun,
1415+ We've no less days to sing God's praise,
1416+ Than when we first begun.
1417+
1418+
1419+John Newton (d. 1807)
1420+
1421+Reproduced from Singing the Faith Electronic Words Edition, number 2 - or not as this is a hand made test file
1422
1423=== added file 'tests/resources/songs/singingthefaith/hints/STF001.json'
1424--- tests/resources/songs/singingthefaith/hints/STF001.json 1970-01-01 00:00:00 +0000
1425+++ tests/resources/songs/singingthefaith/hints/STF001.json 2019-09-04 16:19:42 +0000
1426@@ -0,0 +1,13 @@
1427+{
1428+ "title": "STF001 - Amazing Grace! how sweet the sound!",
1429+ "authors": [
1430+ "John Newton (d. 1807)"
1431+ ],
1432+ "verse_order_list": ["v1"],
1433+ "verses": [
1434+ [
1435+ "Amazing Grace! how sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.",
1436+ "v"
1437+ ]
1438+ ]
1439+}
1440
1441=== added file 'tests/resources/songs/singingthefaith/hints/STF002.json'
1442--- tests/resources/songs/singingthefaith/hints/STF002.json 1970-01-01 00:00:00 +0000
1443+++ tests/resources/songs/singingthefaith/hints/STF002.json 2019-09-04 16:19:42 +0000
1444@@ -0,0 +1,29 @@
1445+{
1446+ "title": "STF002 - Amazing Grace! how sweet the sound!",
1447+ "authors": [
1448+ "John Newton (d. 1807)"
1449+ ],
1450+ "verse_order_list": ["v1", "v2", "v3", "v4", "v5"],
1451+ "verses": [
1452+ [
1453+ "Amazing Grace! how sweet the sound!\nThat saved a wretch like me!\nI once was lost, but now am found;\nWas blind, but now I see.",
1454+ "v"
1455+ ],
1456+ [
1457+ "'Twas grace that taught my heart to fear,\nAnd grace my fears relieved.\nHow precious did that grace appear,\nThe hour I first believed.",
1458+ "v"
1459+ ],
1460+ [
1461+ "The Lord has promised good to me,\nHis Word my hope secures.\nHe will my shield and portion be\nAs long as life endures.",
1462+ "v"
1463+ ],
1464+ [
1465+ "Thro' many dangers, toils and snares\nI have already come.\n'Tis grace that brought me safe thus far,\nAnd grace will lead me home.",
1466+ "v"
1467+ ],
1468+ [
1469+ "When we've been there ten thousand years,\nBright shining as the sun,\nWe've no less days to sing God's praise,\nThan when we first begun.",
1470+ "v"
1471+ ]
1472+ ]
1473+}
1474
1475=== added file 'tests/resources/songs/singingthefaith/hints/singingthefaith-hints.tag'
1476--- tests/resources/songs/singingthefaith/hints/singingthefaith-hints.tag 1970-01-01 00:00:00 +0000
1477+++ tests/resources/songs/singingthefaith/hints/singingthefaith-hints.tag 2019-09-04 16:19:42 +0000
1478@@ -0,0 +1,5 @@
1479+Tag-STFHints-version: 1.0
1480+Version: 2
1481+SongbookNumberInTitle: True
1482+End:
1483+