Merge lp:~acoconut/systers/flexible_essays into lp:systers

Proposed by Ana Cutillas
Status: Superseded
Proposed branch: lp:~acoconut/systers/flexible_essays
Merge into: lp:systers
Diff against target: 286 lines (+131/-28)
5 files modified
Mailman/Cgi/admindb.py (+10/-7)
Mailman/Cgi/listinfo.py (+20/-7)
Mailman/Cgi/subscribe.py (+46/-7)
Mailman/DlistUtils.py (+54/-1)
bin/rmlist (+1/-6)
To merge this branch: bzr merge lp:~acoconut/systers/flexible_essays
Reviewer Review Type Date Requested Status
Robin J Pending
beachbrake Pending
Review via email: mp+118224@code.launchpad.net

Description of the change

I added the functionality to show the pending subscriptions with the new types of essays (in admindb.py)

I fixed the bug found when users tried to write an answer longer than what the database holds (and changed the database schema so it holds up to 1500 characters for the essay answers)

I fixed what happens when a user tries to subscribe twice while their application is being reviewed by making changes to the storm function and counting the number of answers with that question ID and email address. We will have to go through this approach again when we have different form versions, because it will probably not work. I also had to modify subscribe.py, check the return value of the function that writes to the database and give an appropriate error message.

To post a comment you must log in.
Revision history for this message
Robin J (robin-jeffries) wrote :

Ahh, now I understand why we can never make sense of the order the questions come in. Could you change it so they are ordered by subscribe time? (maybe we have to add that field? Or does the database know when the item is entered?) I have always considered the ordering to be random, and when you have "saved" some items to process later, they get reordered and you can't tell the old ones from the new ones.

Also change the comment about modifying this to add essays, since it's not an essay any more. Something like "the answers to the questions asked on the subscribe page?

I think you need to rethink the error checking in subscribe.py First, it's going to confuse the next person who reads your code when you say if len(answer)<1500, since that only applies to text boxes - a checkbox can't be that big. So a branch that has different logic for textboxes than checkboxes will make it clearer. Also, this error message

 results.append(_('At least one of your answers is too long. Your answers must be less than 1500 characters long.'))

is not very use friendly. Tell them which question is too long. You already have info about the questions -- use the long text. (Suppose there are 2 checkboxes and one is under 1500 chars, but not by a lot and the other is over. That will drive the subscriber crazy trying to figure out which ones needs to change)

For results = _("""There is a pending subscription with this email.""")

how about adding "You may not subscribe a second time. The moderator will contact you if there are any questions about the answers you gave originally."

Can you look up and see if there is a way to set a given change (e.g., one that has been approved) as the baseline, so that I can look at only new changes. It's hard to review this stuff when the diffs don't give me much information about what you changed since I last looked at it.

Revision history for this message
Ana Cutillas (acoconut) wrote :

On Mon, Aug 6, 2012 at 3:23 AM, Robin J <email address hidden> wrote:

> Ahh, now I understand why we can never make sense of the order the
> questions come in. Could you change it so they are ordered by subscribe
> time? (maybe we have to add that field? Or does the database know when
> the item is entered?) I have always considered the ordering to be random,
> and when you have "saved" some items to process later, they get reordered
> and you can't tell the old ones from the new ones.
>

I don't understand what you mean here...

Right now the database stores the form_question_id (PK), the answer, the
email, the question version, the question id (these look like "talk",
"woman", etc.) and the status (accepted, pending, rejected).

And they are stored in the database in the same order they have in the
form. So I think the cgi module puts the answers in a list in the same
order they have in the form. Then, I read them in subscribe.py and put them
in a new list, in the same order. And then, a function in DlistUtils.py is
called for every answer (still in order) and this functions stores the
answers in the database. I have a few mock subscriptions in the database
and they all have the same order.

> Also change the comment about modifying this to add essays, since it's not
> an essay any more. Something like "the answers to the questions asked on
> the subscribe page?
>
> I think you need to rethink the error checking in subscribe.py First,
> it's going to confuse the next person who reads your code when you say if
> len(answer)<1500, since that only applies to text boxes - a checkbox can't
> be that big. So a branch that has different logic for textboxes than
> checkboxes will make it clearer. Also, this error message
>
> results.append(_('At least one of your answers is too long. Your answers
> must be less than 1500 characters long.'))
>
> is not very use friendly. Tell them which question is too long. You
> already have info about the questions -- use the long text. (Suppose there
> are 2 checkboxes and one is under 1500 chars, but not by a lot and the
> other is over. That will drive the subscriber crazy trying to figure out
> which ones needs to change)
>
> For results = _("""There is a pending subscription with this email.""")
>
> how about adding "You may not subscribe a second time. The moderator will
> contact you if there are any questions about the answers you gave
> originally."
>
> Can you look up and see if there is a way to set a given change (e.g., one
> that has been approved) as the baseline, so that I can look at only new
> changes. It's hard to review this stuff when the diffs don't give me much
> information about what you changed since I last looked at it.
>

Yeah, it's hard for me to understand what part of the code you're talking
about too. I keep clicking resubmit proposal, maybe I should just send a
new one? I'll give it a try and let's see what that gives us.

I think I fixed everything you said.

> --
> https://code.launchpad.net/~acoconut/systers/flexible_essays/+merge/118224
> You are the owner of lp:~acoconut/systers/flexible_essays.
>

lp:~acoconut/systers/flexible_essays updated
97. By Ana Cutillas <email address hidden>

Made the changes suggested by Robin: concerning error messages and
asserting the size of the answers.

Added the code to mark answers as accepted in the database when a user
is accepted.

Revision history for this message
Robin J (robin-jeffries) wrote :

On Tue, Aug 7, 2012 at 3:11 AM, Ana Cutillas <email address hidden> wrote:

> On Mon, Aug 6, 2012 at 3:23 AM, Robin J <email address hidden> wrote:
>
> > Ahh, now I understand why we can never make sense of the order the
> > questions come in. Could you change it so they are ordered by subscribe
> > time? (maybe we have to add that field? Or does the database know when
> > the item is entered?) I have always considered the ordering to be
> random,
> > and when you have "saved" some items to process later, they get reordered
> > and you can't tell the old ones from the new ones.
> >
>
> I don't understand what you mean here...
>

In admindb.py, there is the comment
  # Alphabetical order by email address

This is how the information is shown on the admindb page. So I'm not
talking about the order of the columns but the order of the rows.

It looks like the issue is that the key is the email address (see the sort
statement a bit lower). You would need to keep track of an approximate
submission time and sort by that. Don't worry about that right now, but
put it on your list of things to do after GSOC is over.

lp:~acoconut/systers/flexible_essays updated
98. By Ana Cutillas <email address hidden>

Changed the logic in the checking of answers in subscribe.py
Cleaned up DlistUtils.py

99. By Ana Cutillas <email address hidden>

Added support for setting answers to rejected in the database.

Unmerged revisions

99. By Ana Cutillas <email address hidden>

Added support for setting answers to rejected in the database.

98. By Ana Cutillas <email address hidden>

Changed the logic in the checking of answers in subscribe.py
Cleaned up DlistUtils.py

97. By Ana Cutillas <email address hidden>

Made the changes suggested by Robin: concerning error messages and
asserting the size of the answers.

Added the code to mark answers as accepted in the database when a user
is accepted.

96. By Ana Cutillas <email address hidden>

Added the error message saying that the user hasn't been subscribe to
the list when a user tries to subscribe with a pending subscription.

95. By Ana Cutillas <email address hidden>

Added the functionality to check that the answers aren't longer than
what the database can hold.

94. By Ana Cutillas <email address hidden>

Added changes to show the new essays in the pending subscriptions table.

Handle the possibility of a user trying to subscribe twice before the
subscription is reviewed.

93. By Ana Cutillas <email address hidden>

Removed the unused function get_answer from DlistUtils.py

92. By Ana Cutillas <email address hidden>

Cleaned up DlistUtils and changed the error message in subscribe. Added
comments.

91. By Ana Cutillas <email address hidden>

Cleaned up.

90. By Ana Cutillas <email address hidden>

Cleaned up listinfo.py

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Mailman/Cgi/admindb.py'
2--- Mailman/Cgi/admindb.py 2009-07-31 10:49:01 +0000
3+++ Mailman/Cgi/admindb.py 2012-08-07 10:41:20 +0000
4@@ -285,11 +285,13 @@
5 form.AddItem(Center(Header(2, _('Subscription Requests'))))
6 table = Table(border=2)
7 # The Essay is there to support subscription with essay, along with below references to "essay"
8- table.AddRow([Center(Bold(_('Address/name'))),
9+ table_headers = ([Center(Bold(_('Address/name'))),
10 Center(Bold(_('Your decision'))),
11- Center(Bold(_('Reason for refusal'))),
12- Center(Bold(_('Essay')))
13+ Center(Bold(_('Reason for refusal')))
14 ])
15+ for question in mlist.questions:
16+ table_headers.append(Center(Bold(question[0])))
17+ table.AddRow(table_headers)
18 # Alphabetical order by email address
19 byaddrs = {}
20 for id in pendingsubs:
21@@ -321,11 +323,12 @@
22 # While the address may be a unicode, it must be ascii
23 paddr = addr.encode('us-ascii', 'replace')
24 # Added by Systers to support essay feature. Lets the list admin see the essay the subscriber has entered.
25- table.AddRow(['%s<br><em>%s</em>' % (paddr, Utils.websafe(fullname)),
26+ content_row = (['%s<br><em>%s</em>' % (paddr, Utils.websafe(fullname)),
27 radio,
28- TextBox('comment-%d' % id, size=40),
29- essay
30- ])
31+ TextBox('comment-%d' % id, size=40)])
32+ for answer in essay:
33+ content_row.append(answer[1])
34+ table.AddRow(content_row)
35 num += 1
36 if num > 0:
37 form.AddItem(table)
38
39=== modified file 'Mailman/Cgi/listinfo.py'
40--- Mailman/Cgi/listinfo.py 2009-08-14 15:41:17 +0000
41+++ Mailman/Cgi/listinfo.py 2012-08-07 10:41:20 +0000
42@@ -22,6 +22,7 @@
43 import os
44 import cgi
45
46+from Mailman import DlistUtils
47 from Mailman import mm_cfg
48 from Mailman import Utils
49 from Mailman import MailList
50@@ -36,7 +37,6 @@
51 i18n.set_language(mm_cfg.DEFAULT_SERVER_LANGUAGE)
52
53
54-
55
56 def main():
57 parts = Utils.GetPathPieces()
58 if not parts:
59@@ -62,7 +62,6 @@
60 list_listinfo(mlist, language)
61
62
63-
64
65 def listinfo_overview(msg=''):
66 # Present the general listinfo overview
67 hostname = Utils.get_domain()
68@@ -152,7 +151,6 @@
69 print doc.Format()
70
71
72-
73
74 def list_listinfo(mlist, lang):
75 # Generate list specific listinfo
76 doc = HeadlessDocument()
77@@ -189,10 +187,25 @@
78
79 # If activated, the content and presence of questions and a field to answer them. Added by Systers.
80 try:
81- if mlist.essay_enabled:
82- replacements['<mm-questions>'] = """<TR>
83- <TD COLSPAN="3">""" + mlist.questions
84- replacements['<mm-essay-box>'] = TextArea('essay', '', rows=8, cols=60).Format() + """ </TD></TR>"""# Added to support subscribe essays
85+ if mlist.essay_enabled: # Added to support subscribe essays
86+ html_questions = []
87+ for question in mlist.questions:
88+ question_name = "'"+question[1]+"'"
89+ html_questions.append("""<TR>
90+ <TD COLSPAN="3">""" + question[0] + """</TD>""")
91+ if (question[2] == 'smallbox'):
92+ html_questions.append("""<TD>""" + TextArea(question_name,\
93+ '', rows=1, cols=60).Format() + """</TD></TR>""")
94+ elif (question[2] == 'largebox'):
95+ html_questions.append("""<TD>""" + TextArea(question_name, \
96+ '', rows=8, cols=60).Format() + """</TD></TR>""")
97+ elif (question[2] == 'checkbox'):
98+ #TODO (Ana Cutillas): consider creating a class like TextArea for checkboxes?
99+ html_questions.append("""<TD><input type='checkbox' name='""" \
100+ + question[1] + """' value =' """\
101+ + question[1] + """'></TD></TR>""")
102+ replacements['<mm-questions>'] = ''.join(html_questions)
103+ replacements['<mm-essay-box>'] = ''
104 else:
105 replacements['<mm-questions>'] = ''
106 replacements['<mm-essay-box>'] = ''
107
108=== modified file 'Mailman/Cgi/subscribe.py'
109--- Mailman/Cgi/subscribe.py 2009-08-14 15:41:17 +0000
110+++ Mailman/Cgi/subscribe.py 2012-08-07 10:41:20 +0000
111@@ -26,6 +26,7 @@
112 from Mailman import MailList
113 from Mailman import Errors
114 from Mailman import i18n
115+from Mailman import DlistUtils # to write to the database
116 from Mailman import Message
117 from Mailman.DlistUserDesc import DlistUserDesc # to support subscribe essays
118 from Mailman.htmlformat import *
119@@ -135,14 +136,42 @@
120 os.environ.get('REMOTE_ADDR',
121 'unidentified origin'))
122
123- # Added to support mailing lists that require essays to join
124+ # Added to support mailing lists that require answers to questions to join
125 if mlist.essay_enabled:
126- essay = cgidata.getvalue('essay', '')
127- if not essay: # Check for empty essays.
128- if not error_msg:
129- print_error(mlist, doc)
130- error_msg = True
131- results.append(_('To subscribe to this list, you must answer the questions asked directly above the large text box.'))
132+ # We need to retrieve the questions to be able to
133+ # retrieve the content from cgi.
134+ essay = []
135+ # This implementation allows for checkboxes to be left unchecked but text boxes must be filled.
136+ for question in mlist.questions:
137+ # A question is a tuple that contains this information:
138+ # (question, question_name, type_of_box, format_info, position, version)
139+ answer = cgidata.getvalue(question[1], '')
140+ if (answer):
141+ # The question has been answered
142+ if (question[2] != 'checkbox'):
143+ if (len(answer) <= 1500):
144+ # In the database we have set the essay max length to 1500
145+ essay.append((question[1], answer, question[5]))
146+ else:
147+ if not error_msg:
148+ print_error(mlist, doc)
149+ error_msg = True
150+ results_error = "Your answer to the question \"" + question[0] + "\" is too long. Your answers must be less than 1500 characters long."
151+ results.append(_(results_error))
152+ else:
153+ # The answer is a checked checkbox
154+ essay.append((question[1], "True", question[5]))
155+ else:
156+ if (question[2] != 'checkbox'):
157+ # If the answer is empty and it's not a checkbox
158+ # the user left fields empty
159+ if not error_msg:
160+ print_error(mlist, doc)
161+ error_msg = True
162+ results.append(_('To subscribe to this list, you must answer all the questions asked on this page.'))
163+ else:
164+ # If the answer is an empty checkbox, the user didn't check it
165+ essay.append((question[1], 'False', question[5]))
166 else: # has to be here, else error message when calling DlistUserDesc
167 essay = cgidata.getvalue('essay', 'None')
168
169@@ -215,6 +244,11 @@
170 may have to be first confirmed by you via email, or approved by the list
171 moderator. If confirmation is required, you will soon get a confirmation
172 email which contains further instructions.""")
173+ for answer in essay:
174+ form = DlistUtils.FormQuestion(mlist)
175+ form_question_id = form.recordAnswer(email, answer[0], answer[1], answer[2])
176+ # If form_question_id is -1, the user has a pending subscription
177+ # An appropriate message is giving to the user later
178
179 try:
180 userdesc = DlistUserDesc(email, fullname, password, digest, lang, essay)
181@@ -303,6 +337,11 @@
182 else:
183 results = _("""\
184 You have been successfully subscribed to the %(realname)s mailing list.""")
185+ if (form_question_id == -1):
186+ if not error_msg:
187+ print_error(mlist, doc)
188+ error_msg = True
189+ results = _("""There is a pending subscription with this email. You may not subscribe a second time. The moderator will contact you if there are any questions about the answers you gave originally.""")
190 # Show the results
191 ## Not sure why this is needed
192 if results == []:
193
194=== modified file 'Mailman/DlistUtils.py'
195--- Mailman/DlistUtils.py 2010-01-31 20:29:48 +0000
196+++ Mailman/DlistUtils.py 2012-08-07 10:41:20 +0000
197@@ -248,6 +248,12 @@
198 command = "count = (self.store.find(Subscriber,Subscriber.mailman_key == unicode(key,'utf-8'))).count()\n"
199 if DEBUG_MODE:
200 syslog('info', 'DlistUtils(subscribeToList):Executing query:\n%s', command)
201+
202+ unicode_key = unicode(key, "utf-8")
203+ count_essay = (self.store.find(FormQuestion, FormQuestion.user_email == unicode_key)).count()
204+ for i in range(count_essay):
205+ self.store.find(FormQuestion, FormQuestion.user_email == unicode_key, FormQuestion.status == u"pending").set(status = u"accepted")
206+
207 # First see if member is subscribed with deleted field = false
208 count = (self.store.find(Subscriber,Subscriber.mailman_key == unicode(key,"utf-8"))).count()
209
210@@ -255,7 +261,7 @@
211 command = "self.store.find(Subscriber,Subscriber.mailman_key == unicode(key,'utf-8')).set(deleted = False)"
212 if DEBUG_MODE:
213 syslog('info', 'DlistUtils(subscribeToList):Executing query:\n%s', command)
214- self.store.find(Subscriber,Subscriber.mailman_key == unicode(key,"utf-8")).set(deleted = False)
215+ self.store.find(Subscriber,Subscriber.mailman_key == unicode_key).set(deleted = False)
216 else:
217 # format is text-only (1) by default
218 command = "subscriber = self.store.add(Subscriber())\nsubscriber.mailman_key = unicode(key,'utf-8')\nsubscriber.preference = 1\nsubscriber.deleted = False\nsubscriber.format = 1\nsubscriber.suppress = 0"
219@@ -289,6 +295,53 @@
220 self.store.find(Alias,Alias.subscriber_id == obsolete_id).set(subscriber_id = new_id)
221 self.store.find(Subscriber,Subscriber.subscriber_id == obsolete_id).remove()
222
223+class FormQuestion(object):
224+ __storm_table__ = "form_question"
225+ form_question_id = Int(primary = True,default = AutoReload)
226+ content = Unicode()
227+ user_email = Unicode()
228+ version = Int()
229+ question_id = Unicode()
230+ status = Unicode()
231+
232+ def __init__(self,mlist):
233+ self.mlist = mlist
234+ self.database = getConn(mlist)
235+
236+ @decfunc
237+ def recordAnswer(self, addr, question, answer, version):
238+ """Save the answer for a question of the subscribe form, returning its id."""
239+ subscriber = Subscriber(self.mlist)
240+ userID = addr.decode('utf-8')
241+ question_unicode = question.decode('utf-8')
242+ count = (self.store.find(FormQuestion, FormQuestion.user_email == userID, FormQuestion.question_id == question_unicode)).count()
243+ if count:
244+ # The user has already tried to subscribe.
245+ return -1
246+ status = u"pending"
247+ question_unicode = question.decode('utf-8')
248+ answer_unicode = answer.decode('utf-8')
249+ #form_question_id has autoreload set, its value will be serially updated in database
250+ self.user_email = userID
251+ self.content = answer_unicode
252+ self.version = version
253+ self.question_id = question_unicode
254+ self.status = status
255+ self.store.add(self)
256+ formQuestionID = self.store.find(FormQuestion).max(FormQuestion.form_question_id)
257+ if DEBUG_MODE:
258+ syslog('info', 'Result of query(formQuestionID) is: %s\n', formQuestionID)
259+ return formQuestionID
260+
261+ @decfunc
262+ def acceptAnswer(self, addr):
263+ """Sets the status of the answers to accepted"""
264+ userID = addr.decode('utf-8')
265+ count = (self.store.find(FormQuestion, FormQuestion.user_email == userID).count())
266+ for i in range(count):
267+ self.store.find(FormQuestion, FormQuestion.user_email == userID).set(status = u"accepted")
268+
269+
270 class Message(object):
271 __storm_table__ = "message"
272 message_id = Int(primary = True,default = AutoReload)
273
274=== modified file 'bin/rmlist'
275--- bin/rmlist 2009-07-31 10:49:01 +0000
276+++ bin/rmlist 2012-08-07 10:41:20 +0000
277@@ -152,12 +152,7 @@
278
279 # Remove the database in PostgreSQL if it was a dynamic list. Added by Systers.
280 try:
281- conn = DlistUtils.getConn(None)
282- old_iso_lvl = conn.isolation_level
283- conn.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
284- cursor = conn.cursor()
285- cursor.execute('DROP DATABASE "%s"' % listname)
286- conn.set_isolation_level(old_iso_lvl)
287+ DlistUtils.remove_dlist(listname)
288 except:
289 syslog('error', 'Tried to remove the database for %s, no such database (ok if it was a non dlist). If it was a dlist, the postgreSQL database for this list needs to be removed manually.' % listname)
290

Subscribers

People subscribed via source and target branches