Merge lp:~oubiwann/ubuntu-accomplishments-system/946850-twisted-app into lp:~jonobacon/ubuntu-accomplishments-system/trophyinfo

Proposed by Duncan McGreggor
Status: Merged
Merge reported by: Jono Bacon
Merged at revision: not available
Proposed branch: lp:~oubiwann/ubuntu-accomplishments-system/946850-twisted-app
Merge into: lp:~jonobacon/ubuntu-accomplishments-system/trophyinfo
Diff against target: 1248 lines (+504/-345)
7 files modified
accomplishments/daemon/api.py (+276/-279)
accomplishments/daemon/app.py (+36/-11)
accomplishments/daemon/dbusapi.py (+46/-38)
accomplishments/daemon/service.py (+80/-0)
accomplishments/gui/TrophyinfoWindow.py (+1/-1)
accomplishments/util/__init__.py (+37/-1)
bin/daemon (+28/-15)
To merge this branch: bzr merge lp:~oubiwann/ubuntu-accomplishments-system/946850-twisted-app
Reviewer Review Type Date Requested Status
Jono Bacon Approve
Review via email: mp+98325@code.launchpad.net

Description of the change

Okay, this branch provides a twistd daemon. The wiki will need to be updated for usage:

To run the daemon under non-development conditions (e.g., from an rc script) one needs to invoke the daemon in the following manner:

 $ twistd -y ./bin/daemon

This will write a log file and a pid file to the working directory.

From an rc script, you'd (obviously) need to give the full path and have the accomplishments code installed on the Python path for the version that the rc script would be using. Also, for an rc script, you will probably want to pass options for the pid file and log file directories, and maybe even uid/guid options.

When developing, one can use the following so that the daemon is not daemonized, but instead writes it log file to stdout:

 $ twistd -noy ./bin/daemon

Usage notes aside, this branch makes some big changes. First and foremost, it introduces a set of Twisted application services and nests them so that we can have fine-grained control over what gets started, stopped, etc., and what does it first, what depends on another thing, etc.

The second big change is the use of an asyncapi attribute to reference all the async calls. An AsyncAPI class was created to keep things straight (what calls depend upon Twisted and deferreds, and which ones don't). The intent with this move is to continue this trend with the other code (in future branches): a ConfigAPI and configapi attribute, ExtraInfoAPI and extrainfoapi attribute, etc.

The other thing to keep in mind with this branch is that it adds lots of TODOs (look for XXX comments) as inline comments and makes recommendations on refactoring a significant portion of the api.py code. In particular, all blocking calls that are going to be made in functions/methods that return deferreds should either be converted to async code that utilizes Twisted to accomplish the same thing, or be run in a thread with deferToThread. Otherwise, you will get poor (blocking) performance and not receive the benefits of Twisted.

Another big refactor that needs to happen is breaking the methods up: they are large and usually have more responsibility than they should. Several TODO comments were made that point the direction for this (e.g., "split this out into sync and share methods"),

No attempt to change any of that was done in this branch, as the focus here was simply to get a daemon created that was twistd-friendly.

To post a comment you must log in.
Revision history for this message
Duncan McGreggor (oubiwann) wrote :

Oh, another note for future work: there are a bunch of pyflakes. Install pyflakes (sudo apt-get install pyflakes) and run it against the accomplishments directory. There's lots of stuff to be fixed.

Similarly with the pep8 tool.

Revision history for this message
Jono Bacon (jonobacon) wrote :

Thanks so much, Duncan, for taking the time to work on this. I have a few questions, bearing in mind some of my more limited Twisted knowledge.

 * It looks like I needed to set my PYTHONPATH, is this the case?
 * I see you created AsyncAPI - does this literally just put these async orientated methods into a different class for the purposes of classification/naming, or does this do anything differently?
 * It looks like each of the classes in service.py overload a set of methods in Twisted (startService, stopService etc) and then applicationFactory in app.py gets everything up and running. Is this right?
 * In the notes you say "all blocking calls that are going to be made in functions/methods that return deferreds should either be converted to async code that utilizes Twisted to accomplish the same thing, or be run in a thread with deferToThread" - I thought that by using inline callbacks (as my code does already) this was doing this the Twisted way and not blocking?

Thanks!

   Jono

Revision history for this message
Duncan McGreggor (oubiwann) wrote :
Download full text (3.8 KiB)

On Thu, Mar 22, 2012 at 1:23 PM, Jono Bacon <email address hidden> wrote:
> Thanks so much, Duncan, for taking the time to work on this. I have a few questions, bearing in mind some of my more limited Twisted knowledge.
>
>  * It looks like I needed to set my PYTHONPATH, is this the case?

I was never able to duplicate the issues you had with importing python
files from the working directory. I added a sys.path.insert(0, ".") to
hopefully overcome that for you. If that works, you should be all set.
If not, we'll have to do some trial and error to see what does work.
There should be no need to set any environment variables for this to
work, though.

>  * I see you created AsyncAPI - does this literally just put these async orientated methods into a different class for the purposes of classification/naming

Yup.

The idea is that all the different types of API calls that are mixed
in the Accomplishments class right now would be organized into their
own class and then instantiated like AsyncAPI is (e.g., self.asyncapi
= AsyncAPI()). This is simply to help with code organization,
readability, etc.

>  * It looks like each of the classes in service.py overload a set of methods in Twisted (startService, stopService etc)

It doesn't overload. It simply implements the method defined by the
interface. Each service can potentially do lots of difference things
in its startService and stopService methods -- all different from each
other, since each service is of a different type and has different
responsibilities.

This would be a good place to do any DBus main loop tweaks, if needed,
for instance.

(Btw, Python can't really do classic overloading, since a
method/function can only be defined once in the same scope. However,
you can achieve a mostly similar-seeming result with positional and
named args and a dispatch mechanism. You could also override and then
call the superclass's method, that also gets you something similar to
overloading.)

> and then applicationFactory in app.py gets everything up and running. Is this right?

Almost. applicationFactory creates an application object (with lots of
child components) and assigns that to the variable name "application"
in the .tac file. twistd loads the .tac file, looks for the
"application" variable name, loads the object stored there, and fires
of the chain of startService methods on each child object.

So it's twistd that gets everything up and running. applicationFactory
just applies the configuration to various instantiations and then
passes the application object back.

>  * In the notes you say "all blocking calls that are going to be made in functions/methods that return deferreds should either be converted to async code that utilizes Twisted to accomplish the same thing, or be run in a thread with deferToThread" - I thought that by using inline callbacks (as my code does already) this was doing this the Twisted way and not blocking?

Nope. This is one of the many pitfalls of using inlineCallbacks.

inlineCallbacks is basically a wrapper for deferreds. It doesn't
magically make code non-blocking any more than using deferreds
directly does. There's a famous saying in the Twisted community:
  http://www.f...

Read more...

Revision history for this message
Jono Bacon (jonobacon) wrote :
Download full text (5.0 KiB)

Sorry for the delay, Duncan, I am still on vacation.

On 22 March 2012 15:22, Duncan McGreggor <email address hidden> wrote:
>>  * It looks like I needed to set my PYTHONPATH, is this the case?
>
> I was never able to duplicate the issues you had with importing python
> files from the working directory. I added a sys.path.insert(0, ".") to
> hopefully overcome that for you. If that works, you should be all set.
> If not, we'll have to do some trial and error to see what does work.
> There should be no need to set any environment variables for this to
> work, though.

Thanks, Duncan, I will test again soon.

>>  * I see you created AsyncAPI - does this literally just put these async orientated methods into a different class for the purposes of classification/naming
>
> Yup.
>
> The idea is that all the different types of API calls that are mixed
> in the Accomplishments class right now would be organized into their
> own class and then instantiated like AsyncAPI is (e.g., self.asyncapi
> = AsyncAPI()). This is simply to help with code organization,
> readability, etc.

Makes sense. :-)

>>  * It looks like each of the classes in service.py overload a set of methods in Twisted (startService, stopService etc)
>
> It doesn't overload. It simply implements the method defined by the
> interface. Each service can potentially do lots of difference things
> in its startService and stopService methods -- all different from each
> other, since each service is of a different type and has different
> responsibilities.
>
> This would be a good place to do any DBus main loop tweaks, if needed,
> for instance.
>
> (Btw, Python can't really do classic overloading, since a
> method/function can only be defined once in the same scope. However,
> you can achieve a mostly similar-seeming result with positional and
> named args and a dispatch mechanism. You could also override and then
> call the superclass's method, that also gets you something similar to
> overloading.)

So it looks like each of the different services are set up and each of
these are then used throughout the rest of the API (e.g. when we
run_scripts() this uses the ScriptRunner service). I guess the
startService/stopService pieces are then used to help them start and
stop in a twistd way (incorporating logging etc), right?

>> and then applicationFactory in app.py gets everything up and running. Is this right?
>
> Almost. applicationFactory creates an application object (with lots of
> child components) and assigns that to the variable name "application"
> in the .tac file. twistd loads the .tac file, looks for the
> "application" variable name, loads the object stored there, and fires
> of the chain of startService methods on each child object.

I saw mention of a .tac file in the twistd docs, but I didn't see it
in the code. Is there a file somewhere I should see?

> So it's twistd that gets everything up and running. applicationFactory
> just applies the configuration to various instantiations and then
> passes the application object back.

Gotcha. Btw, you showed me how to start a twistd service, how do I stop it?

Also, do you know if there is a good way to map twistd start/stop to
upstart ...

Read more...

Revision history for this message
Duncan McGreggor (oubiwann) wrote :
Download full text (6.4 KiB)

On Mon, Mar 26, 2012 at 2:01 AM, Jono Bacon <email address hidden> wrote:
> On 22 March 2012 15:22, Duncan McGreggor <email address hidden> wrote:
>>
>>>  * It looks like each of the classes in service.py overload a set of methods in Twisted (startService, stopService etc)
>>
>> It doesn't overload. It simply implements the method defined by the
>> interface. Each service can potentially do lots of difference things
>> in its startService and stopService methods -- all different from each
>> other, since each service is of a different type and has different
>> responsibilities.
>>
>> This would be a good place to do any DBus main loop tweaks, if needed,
>> for instance.
>>
>> (Btw, Python can't really do classic overloading, since a
>> method/function can only be defined once in the same scope. However,
>> you can achieve a mostly similar-seeming result with positional and
>> named args and a dispatch mechanism. You could also override and then
>> call the superclass's method, that also gets you something similar to
>> overloading.)
>
> So it looks like each of the different services are set up and each of
> these are then used throughout the rest of the API (e.g. when we
> run_scripts() this uses the ScriptRunner service). I guess the
> startService/stopService pieces are then used to help them start and
> stop in a twistd way (incorporating logging etc), right?

Yup.

>>> and then applicationFactory in app.py gets everything up and running. Is this right?
>>
>> Almost. applicationFactory creates an application object (with lots of
>> child components) and assigns that to the variable name "application"
>> in the .tac file. twistd loads the .tac file, looks for the
>> "application" variable name, loads the object stored there, and fires
>> of the chain of startService methods on each child object.
>
> I saw mention of a .tac file in the twistd docs, but I didn't see it
> in the code. Is there a file somewhere I should see?

Sorry, I renamed it to not use an extension, since it will be used as
a daemon in a long-running process. It's still a .tac file, though.
(it's in the bin dir)

>> So it's twistd that gets everything up and running. applicationFactory
>> just applies the configuration to various instantiations and then
>> passes the application object back.
>
> Gotcha. Btw, you showed me how to start a twistd service, how do I stop it?

So when run in non-daemonized mode (for development), it's just ^C.

When run in daemon mode, it writes std out to a log file
(configurable; see twistd --help) and the PID of the process to a .pid
file (configurable; see twistd --help). Sys V init scripts typically
generate a file or shell variable themselves then later cat the pid
file when running a kill against it. You'll do the same thing here,
except the pid file has already been created for you.

> Also, do you know if there is a good way to map twistd start/stop to
> upstart start/stop events?

I've never written upstart scripts, but twistd is used in all sorts of
init/rc scripts, similarly to how I outlined above.

Jereb (another core hacker/long-time Twisted guy) did a great write-up
here for plugins:
  https://bitbucket.org/jerub/twisted-plugin-exampl...

Read more...

Revision history for this message
Jono Bacon (jonobacon) wrote :

Thanks for your work on thus Duncan. I think I have my head around this now. I took a good look at it today and fixed the issue with trophy_received and I have just merged this in.

I might drop you a message every so often if I have further Twisted questions if that is OK. Thanks!

review: Approve
Revision history for this message
Duncan McGreggor (oubiwann) wrote :

On Sat, Mar 31, 2012 at 8:06 PM, Jono Bacon <email address hidden> wrote:
> Review: Approve
>
> Thanks for your work on thus Duncan. I think I have my head around this now. I took a good look at it today and fixed the issue with trophy_received and I have just merged this in.
>
> I might drop you a message every so often if I have further Twisted questions if that is OK.

You betcha!

> Thanks!

No prob, bro :-)

d

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'accomplishments/daemon/api.py'
--- accomplishments/daemon/api.py 2012-03-18 16:56:29 +0000
+++ accomplishments/daemon/api.py 2012-03-20 03:24:21 +0000
@@ -12,7 +12,6 @@
12import gobject12import gobject
13import gpgme13import gpgme
14import json14import json
15import logging
16import os15import os
17import pwd16import pwd
18import subprocess17import subprocess
@@ -24,6 +23,7 @@
24from twisted.internet import defer, reactor23from twisted.internet import defer, reactor
25from twisted.internet.protocol import ProcessProtocol24from twisted.internet.protocol import ProcessProtocol
26from twisted.python import filepath25from twisted.python import filepath
26from twisted.python import log
2727
28import xdg.BaseDirectory28import xdg.BaseDirectory
2929
@@ -38,7 +38,7 @@
38import accomplishments38import accomplishments
39from accomplishments import exceptions39from accomplishments import exceptions
40from accomplishments.daemon import dbusapi40from accomplishments.daemon import dbusapi
41from accomplishments.util import get_data_file41from accomplishments.util import get_data_file, SubprocessReturnCodeProtocol
4242
4343
44MATRIX_USERNAME = "openiduser155707"44MATRIX_USERNAME = "openiduser155707"
@@ -46,20 +46,229 @@
46SCRIPT_DELAY = 90046SCRIPT_DELAY = 900
4747
4848
49class SubprocessReturnCodeProtocol(ProcessProtocol):49# XXX the source code needs to be updated to use Twisted async calls better:
50 """50# grep the source code for any *.asyncapi.* references, and if they return
51 """51# deferreds, adjust them to use callbacks
52 def connectionMade(self):52class AsyncAPI(object):
53 self.returnCodeDeferred = defer.Deferred()53 """
5454 This class simply organizes all the Twisted calls into a single location
55 def processEnded(self, reason):55 for better readability and separation of concerns.
56 self.returnCodeDeferred.callback(reason.value.exitCode)56 """
5757 def __init__(self, parent):
58 def outReceived(self, data):58 self.parent = parent
59 print data59
6060 @staticmethod
61 def errReceived(self, data):61 def run_a_subprocess(command):
62 print data62 log.msg("Running subprocess command: " + str(command))
63 pprotocol = SubprocessReturnCodeProtocol()
64 reactor.spawnProcess(pprotocol, command[0], command, env=os.environ)
65 return pprotocol.returnCodeDeferred
66
67 # XXX let's rewrite this to use deferreds explicitly
68 @defer.inlineCallbacks
69 def wait_until_a_sig_file_arrives(self):
70 path, info = yield self.parent.sd.wait_for_signals(
71 signal_ok="DownloadFinished",
72 success_filter=lambda path,
73 info: path.startswith(self.trophies_path)
74 and path.endswith(".asc"))
75 log.msg("Trophy signature recieved...")
76 accomname = os.path.splitext(os.path.splitext(
77 os.path.split(path)[1])[0])[0]
78 data = self.parent.listAccomplishmentInfo(accomname)
79 iconpath = os.path.join(
80 self.parent.accomplishments_path,
81 data[0]["application"],
82 "trophyimages",
83 data[0]["icon"])
84
85 item = os.path.split(path)[1][:-11]
86 app = os.path.split(os.path.split(path)[0])[1]
87 data = self.parent.listAccomplishmentInfo(item)
88
89 if self.parent.scriptrun_total == len(self.parent.scriptrun_results):
90 self.parent.show_unlocked_accomplishments()
91
92 if self.parent.show_notifications == True and pynotify and (
93 pynotify.is_initted() or pynotify.init("icon-summary-body")):
94 self.parent.service.trophy_received("foo")
95 trophy_icon_path = "file://%s" % os.path.realpath(
96 os.path.join(
97 os.path.split(__file__)[0],
98 "trophy-accomplished.svg"))
99 n = pynotify.Notification(
100 "You have accomplished something!", data[0]["title"], iconpath)
101 n.show()
102
103 self.wait_until_a_sig_file_arrives()
104 #reload_trophy_corresponding_to_sig_file(path)
105
106 # XXX let's rewrite this to use deferreds explicitly
107 @defer.inlineCallbacks
108 def register_trophy_dir(self, trophydir):
109 """
110 Creates the Ubuntu One share for the trophydir and offers it to the
111 server. Returns True if the folder was successfully shared, False if
112 not.
113 """
114 timeid = str(time.time())
115 log.msg("Registering Ubuntu One share directory: " + trophydir)
116
117 folder_list = yield self.parent.sd.get_folders()
118 folder_is_synced = False
119 for folder in folder_list:
120 if folder["path"] == trophydir:
121 folder_is_synced = True
122 break
123 if not folder_is_synced:
124 # XXX let's breack this out into a separate sync'ing method
125 log.msg(
126 "...the '%s' folder is not synced with the Matrix" % trophydir)
127 log.msg("...creating the share folder on Ubuntu One")
128 self.parent.sd.create_folder(trophydir)
129
130 success_filter = lambda info: info["path"] == trophydir
131 info = yield self.parent.sd.wait_for_signals(
132 signal_ok='FolderCreated', success_filter=success_filter)
133
134 self.parent.sd.offer_share(
135 trophydir, MATRIX_USERNAME, LOCAL_USERNAME + " Trophies Folder"
136 + " (" + timeid + ")", "Modify")
137 log.msg(
138 "...share has been offered (" + trophydir + "" + ", "
139 + MATRIX_USERNAME + ", " + LOCAL_USERNAME + ")")
140 return
141
142 log.msg("...the '%s' folder is already synced" % trophydir)
143 # XXX put the following logic into a folders (plural) sharing method
144 log.msg("... now checking whether it's shared")
145 shared_list = yield self.parent.sd.list_shared()
146 folder_is_shared = False
147 shared_to = []
148 for share in shared_list:
149 # XXX let's break this out into a separate share-tracking method
150 if share["path"] == trophydir:
151 log.msg("...the folder is already shared.")
152 folder_is_shared = True
153 shared_to.append("%s (%s)" % (
154 share["other_visible_name"], share["other_username"]))
155 if not folder_is_shared:
156 # XXX let's break this out into a separate folder-sharing method
157 log.msg("...the '%s' folder is not shared" % trophydir)
158 self.parent.sd.offer_share(
159 trophydir, MATRIX_USERNAME, LOCAL_USERNAME + " Trophies Folder"
160 + " (" + timeid + ")", "Modify")
161 log.msg("...share has been offered (" + trophydir + "" + ", "
162 + MATRIX_USERNAME + ", " + LOCAL_USERNAME + ")")
163 log.msg("...offered the share.")
164 return
165 else:
166 log.msg("The folder is shared, with: %s" % ", ".join(
167 shared_to))
168 return
169
170 # XXX let's rewrite this to use deferreds explicitly
171 @defer.inlineCallbacks
172 def run_scripts_for_user(self, uid):
173 log.msg("--- Starting Running Scripts ---")
174 timestart = time.time()
175 self.parent.service.scriptrunner_start()
176
177 # Is the user currently logged in and running a gnome session?
178 # XXX use deferToThread
179 username = pwd.getpwuid(uid).pw_name
180 try:
181 # XXX since we're using Twisted, let's use it here too and use the
182 # deferred-returning call
183 proc = subprocess.check_output(
184 ["pgrep", "-u", username, "gnome-session"]).strip()
185 except subprocess.CalledProcessError:
186 # user does not have gnome-session running or isn't logged in at
187 # all
188 log.msg("No gnome-session process for user %s" % username)
189 return
190 # XXX this is a blocking call and can't be here if we want to take
191 # advantage of deferreds; instead, rewrite this so that the blocking
192 # call occurs in a separate thread (e.g., deferToThread)
193 fp = open("/proc/%s/environ" % proc)
194 try:
195 envars = dict(
196 [line.split("=", 1) for line in fp.read().split("\0")
197 if line.strip()])
198 except IOError:
199 # user does not have gnome-session running or isn't logged in at
200 # all
201 log.msg("No gnome-session environment for user %s" % username)
202 return
203 fp.close()
204
205 # XXX use deferToThread
206 os.seteuid(uid)
207
208 required_envars = ['DBUS_SESSION_BUS_ADDRESS']
209 env = dict([kv for kv in envars.items() if kv[0] in required_envars])
210 # XXX use deferToThread
211 oldenviron = os.environ
212 os.environ.update(env)
213 # XXX note that for many of these deferredToThread changes, we can put
214 # them all in a DeferredList and once they're all done and we have the
215 # results for all of them, a callback can be fired to continue.
216
217 # XXX this next call, a DBus check, happens in the middle of this
218 # method; it would be better if this check was done at a higher level,
219 # for instance, where this class is initiated: if the daemon isn't
220 # registered at the time of instantiation, simply abort then instead of
221 # making all the way here and then aborting. (Note that moving this
222 # check to that location will also eliminate an obvious circular
223 # import.)
224 if not dbusapi.daemon_is_registered():
225 return
226
227 # XXX all parent calls should be refactored out of the AsyncAPI class
228 # to keep the code cleaner and the logic more limited to one particular
229 # task
230 accoms = self.parent.listAllAvailableAccomplishmentsWithScripts()
231 totalscripts = len(accoms)
232 self.parent.scriptrun_total = totalscripts
233 log.msg("Need to run (%d) scripts" % totalscripts)
234
235 scriptcount = 1
236 for accom in accoms:
237 msg = "%s/%s: %s" % (scriptcount, totalscripts, accom["_script"])
238 log.msg(msg)
239 exitcode = yield self.run_a_subprocess([accom["_script"]])
240 if exitcode == 0:
241 self.parent.scriptrun_results.append(
242 str(accom["application"]) + "/"
243 + str(accom["accomplishment"]))
244 self.parent.accomplish(
245 accom["application"], accom["accomplishment"])
246 log.msg("...Accomplished")
247 elif exitcode == 1:
248 self.parent.scriptrun_results.append(None)
249 log.msg("...Not Accomplished")
250 elif exitcode == 2:
251 self.parent.scriptrun_results.append(None)
252 log.msg("....Error")
253 elif exitcode == 4:
254 self.parent.scriptrun_results.append(None)
255 log.msg("...Could not get launchpad email")
256 else:
257 self.parent.scriptrun_results.append(None)
258 log.msg("...Error code %d" % exitcode)
259 scriptcount = scriptcount + 1
260
261 os.environ = oldenviron
262
263 # XXX eventually the code in this method will be rewritten using
264 # deferreds; as such, we're going to have to be more clever regarding
265 # timing things...
266 timeend = time.time()
267 timefinal = round((timeend - timestart), 2)
268
269 log.msg(
270 "--- Completed Running Scripts in %.2f seconds---" % timefinal)
271 self.parent.service.scriptrunner_finish()
63272
64273
65class Accomplishments(object):274class Accomplishments(object):
@@ -82,9 +291,9 @@
82 self.scriptrun_results = []291 self.scriptrun_results = []
83 self.depends = []292 self.depends = []
84 self.processing_unlocked = False293 self.processing_unlocked = False
294 self.asyncapi = AsyncAPI(self)
85295
86 # create config / data dirs if they don't exist296 # create config / data dirs if they don't exist
87
88 self.dir_config = os.path.join(297 self.dir_config = os.path.join(
89 xdg.BaseDirectory.xdg_config_home, "accomplishments")298 xdg.BaseDirectory.xdg_config_home, "accomplishments")
90 self.dir_data = os.path.join(299 self.dir_data = os.path.join(
@@ -101,73 +310,26 @@
101 if not os.path.exists(self.dir_cache):310 if not os.path.exists(self.dir_cache):
102 os.makedirs(self.dir_cache)311 os.makedirs(self.dir_cache)
103312
104 # set up logging313 log.msg(
105 logdir = os.path.join(self.dir_cache, "logs")
106
107 if not os.path.exists(logdir):
108 os.makedirs(logdir)
109
110 #self.logging = logging
111 logging.basicConfig(
112 filename=(os.path.join(logdir, 'daemon.log')), level=logging.INFO)
113
114 now = datetime.datetime.now()
115 logging.info(
116 "------------------- Ubuntu Accomplishments Daemon Log - %s "314 "------------------- Ubuntu Accomplishments Daemon Log - %s "
117 "-------------------", str(now))315 "-------------------", str(datetime.datetime.now()))
118316
119 self._loadConfigFile()317 self._loadConfigFile()
120318
121 print "Accomplishments path: " + self.accomplishments_path319 log.msg("Accomplishments path: " + self.accomplishments_path)
122 print "Scripts path: " + self.scripts_path320 log.msg("Scripts path: " + self.scripts_path)
123 print "Trophies path: " + self.trophies_path321 log.msg("Trophies path: " + self.trophies_path)
124322
125 self.show_notifications = show_notifications323 self.show_notifications = show_notifications
126 logging.info("Connecting to Ubuntu One")324 log.msg("Connecting to Ubuntu One")
127 self.sd = SyncDaemonTool()325 self.sd = SyncDaemonTool()
128326
129 self.wait_until_a_sig_file_arrives()327 # XXX this wait-until thing should go away; it should be replaced by a
328 # deferred-returning function that has a callback which fires off
329 # generate_all_trophis and schedule_run_scripts...
330 self.asyncapi.wait_until_a_sig_file_arrives()
130 self.generate_all_trophies()331 self.generate_all_trophies()
131332
132 reactor.callLater(5, self.run_scripts, False)
133
134 @defer.inlineCallbacks
135 def wait_until_a_sig_file_arrives(self):
136 path, info = yield self.sd.wait_for_signals(
137 signal_ok="DownloadFinished",
138 success_filter=lambda path,
139 info: path.startswith(self.trophies_path)
140 and path.endswith(".asc"))
141 logging.info("Trophy signature recieved...")
142 accomname = os.path.splitext(os.path.splitext(
143 os.path.split(path)[1])[0])[0]
144 data = self.listAccomplishmentInfo(accomname)
145 iconpath = os.path.join(
146 self.accomplishments_path,
147 data[0]["application"],
148 "trophyimages",
149 data[0]["icon"])
150
151 item = os.path.split(path)[1][:-11]
152 app = os.path.split(os.path.split(path)[0])[1]
153 data = self.listAccomplishmentInfo(item)
154
155 if self.scriptrun_total == len(self.scriptrun_results):
156 self.show_unlocked_accomplishments()
157
158 if self.show_notifications == True and pynotify and (
159 pynotify.is_initted() or pynotify.init("icon-summary-body")):
160 self.service.trophy_received("foo")
161 trophy_icon_path = "file://%s" % os.path.realpath(
162 os.path.join(
163 os.path.split(__file__)[0],
164 "trophy-accomplished.svg"))
165 n = pynotify.Notification(
166 "You have accomplished something!", data[0]["title"], iconpath)
167 n.show()
168
169 self.wait_until_a_sig_file_arrives()
170 #reload_trophy_corresponding_to_sig_file(path)
171333
172 def get_media_file(self, media_file_name):334 def get_media_file(self, media_file_name):
173 media_filename = get_data_file('media', '%s' % (media_file_name,))335 media_filename = get_data_file('media', '%s' % (media_file_name,))
@@ -210,14 +372,14 @@
210 self.depends = []372 self.depends = []
211373
212 def _get_accomplishments_files_list(self):374 def _get_accomplishments_files_list(self):
213 logging.info("Looking for accomplishments files in "375 log.msg("Looking for accomplishments files in "
214 + self.accomplishments_path)376 + self.accomplishments_path)
215 accom_files = os.path.join(self.accomplishments_path,377 accom_files = os.path.join(self.accomplishments_path,
216 "*", "*.accomplishment")378 "*", "*.accomplishment")
217 return glob.glob(accom_files)379 return glob.glob(accom_files)
218380
219 def _load_accomplishment_file(self, f):381 def _load_accomplishment_file(self, f):
220 logging.info("Loading accomplishments file: " + f)382 log.msg("Loading accomplishments file: " + f)
221 config = ConfigParser.RawConfigParser()383 config = ConfigParser.RawConfigParser()
222 config.read(f)384 config.read(f)
223 data = dict(config._sections["accomplishment"])385 data = dict(config._sections["accomplishment"])
@@ -232,7 +394,7 @@
232 Returns True for valid or False for invalid (missing file, bad sig394 Returns True for valid or False for invalid (missing file, bad sig
233 etc).395 etc).
234 """396 """
235 logging.info("Validate trophy: " + str(filename))397 log.msg("Validate trophy: " + str(filename))
236398
237 if os.path.exists(filename):399 if os.path.exists(filename):
238 # the .asc signed file exists, so let's verify that it is correctly400 # the .asc signed file exists, so let's verify that it is correctly
@@ -246,24 +408,24 @@
246 sig = c.verify(signed, None, plaintext)408 sig = c.verify(signed, None, plaintext)
247409
248 if len(sig) != 1:410 if len(sig) != 1:
249 logging.info("...No Sig")411 log.msg("...No Sig")
250 return False412 return False
251413
252 if sig[0].status is not None:414 if sig[0].status is not None:
253 logging.info("...Bad Sig")415 log.msg("...Bad Sig")
254 return False416 return False
255 else:417 else:
256 result = {'timestamp': sig[0].timestamp, 'signer': sig[0].fpr}418 result = {'timestamp': sig[0].timestamp, 'signer': sig[0].fpr}
257 logging.info("...Verified!")419 log.msg("...Verified!")
258 return True420 return True
259 else:421 else:
260 logging.info(".asc does not exist for this trophy")422 log.msg(".asc does not exist for this trophy")
261 return False423 return False
262424
263 logging.info("Verifying trophy signature")425 log.msg("Verifying trophy signature")
264426
265 def _load_trophy_file(self, f):427 def _load_trophy_file(self, f):
266 logging.info("Load trophy file: " + f)428 log.msg("Load trophy file: " + f)
267 config = ConfigParser.RawConfigParser()429 config = ConfigParser.RawConfigParser()
268 config.read(f)430 config.read(f)
269 data = dict(config._sections["trophy"])431 data = dict(config._sections["trophy"])
@@ -272,7 +434,7 @@
272 return data434 return data
273435
274 def listAllAccomplishments(self):436 def listAllAccomplishments(self):
275 logging.info("List all accomplishments")437 log.msg("List all accomplishments")
276 fs = [self._load_accomplishment_file(f) for f in438 fs = [self._load_accomplishment_file(f) for f in
277 self._get_accomplishments_files_list()]439 self._get_accomplishments_files_list()]
278 return fs440 return fs
@@ -351,32 +513,32 @@
351 img = Image.composite(layer, reduced, layer)513 img = Image.composite(layer, reduced, layer)
352 img.save(filecore + "-locked" + filetype)514 img.save(filecore + "-locked" + filetype)
353 except Exception, (msg):515 except Exception, (msg):
354 print msg516 log.msg(msg)
355517
356 def verifyU1Account(self):518 def verifyU1Account(self):
357 # check if this machine has an Ubuntu One account519 # check if this machine has an Ubuntu One account
358 logging.info("Check if this machine has an Ubuntu One account...")520 log.msg("Check if this machine has an Ubuntu One account...")
359 u1auth_response = auth.request(521 u1auth_response = auth.request(
360 url='https://one.ubuntu.com/api/account/')522 url='https://one.ubuntu.com/api/account/')
361 u1email = None523 u1email = None
362 if not isinstance(u1auth_response, basestring):524 if not isinstance(u1auth_response, basestring):
363 u1email = json.loads(u1auth_response[1])['email']525 u1email = json.loads(u1auth_response[1])['email']
364 else:526 else:
365 logging.info("No Ubuntu One account is configured.")527 log.msg("No Ubuntu One account is configured.")
366528
367 if u1email is None:529 if u1email is None:
368 logging.info("...No.")530 log.msg("...No.")
369 logging.info(u1auth_response)531 log.msg(u1auth_response)
370 self.has_u1 = False532 self.has_u1 = False
371 return False533 return False
372 else:534 else:
373 logging.info("...Yes.")535 log.msg("...Yes.")
374 self.has_u1 = True536 self.has_u1 = True
375 return True537 return True
376538
377 def getConfigValue(self, section, item):539 def getConfigValue(self, section, item):
378 """Return a configuration value from the .accomplishments file"""540 """Return a configuration value from the .accomplishments file"""
379 logging.info(541 log.msg(
380 "Returning configuration values for: %s, %s", section, item)542 "Returning configuration values for: %s, %s", section, item)
381 homedir = os.getenv("HOME")543 homedir = os.getenv("HOME")
382 config = ConfigParser.RawConfigParser()544 config = ConfigParser.RawConfigParser()
@@ -395,7 +557,7 @@
395557
396 def setConfigValue(self, section, item, value):558 def setConfigValue(self, section, item, value):
397 """Set a configuration value in the .accomplishments file"""559 """Set a configuration value in the .accomplishments file"""
398 logging.info(560 log.msg(
399 "Set configuration file value in '%s': %s = %s", section, item,561 "Set configuration file value in '%s': %s = %s", section, item,
400 value)562 value)
401 homedir = os.getenv("HOME")563 homedir = os.getenv("HOME")
@@ -415,7 +577,7 @@
415 self._loadConfigFile()577 self._loadConfigFile()
416578
417 def _writeConfigFile(self):579 def _writeConfigFile(self):
418 logging.info("Writing the configuration file")580 log.msg("Writing the configuration file")
419 homedir = os.getenv("HOME")581 homedir = os.getenv("HOME")
420 config = ConfigParser.RawConfigParser()582 config = ConfigParser.RawConfigParser()
421 cfile = self.dir_config + "/.accomplishments"583 cfile = self.dir_config + "/.accomplishments"
@@ -433,10 +595,10 @@
433595
434 self.accomplishments_path = os.path.join(596 self.accomplishments_path = os.path.join(
435 self.accomplishments_path, "accomplishments")597 self.accomplishments_path, "accomplishments")
436 logging.info("...done.")598 log.msg("...done.")
437599
438 def accomplish(self, app, accomplishment_name):600 def accomplish(self, app, accomplishment_name):
439 logging.info(601 log.msg(
440 "Accomplishing something: %s, %s", app, accomplishment_name)602 "Accomplishing something: %s, %s", app, accomplishment_name)
441 accom_file = os.path.join(self.accomplishments_path, app,603 accom_file = os.path.join(self.accomplishments_path, app,
442 "%s.accomplishment" % accomplishment_name)604 "%s.accomplishment" % accomplishment_name)
@@ -481,9 +643,9 @@
481 cp.write(fp)643 cp.write(fp)
482 fp.close()644 fp.close()
483645
484 if not data["needs-signing"] or data["needs-signing"] == False:646 if not data["needs-signing"] or data["needs-signing"] is False:
485 self.trophy_received()647 self.service.trophy_received()
486 if self.show_notifications == True and pynotify and (648 if self.show_notifications is True and pynotify and (
487 pynotify.is_initted() or pynotify.init("icon-summary-body")):649 pynotify.is_initted() or pynotify.init("icon-summary-body")):
488 trophy_icon_path = "file://%s" % os.path.realpath(650 trophy_icon_path = "file://%s" % os.path.realpath(
489 os.path.join(651 os.path.join(
@@ -507,19 +669,19 @@
507 self.has_u1 = True669 self.has_u1 = True
508670
509 if config.read(cfile):671 if config.read(cfile):
510 logging.info("Loading configuration file: " + cfile)672 log.msg("Loading configuration file: " + cfile)
511 if config.get('config', 'accompath'):673 if config.get('config', 'accompath'):
512 self.accomplishments_path = os.path.join(674 self.accomplishments_path = os.path.join(
513 config.get('config', 'accompath'), "accomplishments/")675 config.get('config', 'accompath'), "accomplishments/")
514 logging.info(676 log.msg(
515 "...setting accomplishments path to: "677 "...setting accomplishments path to: "
516 + self.accomplishments_path)678 + self.accomplishments_path)
517 self.scripts_path = os.path.split(679 self.scripts_path = os.path.split(
518 os.path.split(self.accomplishments_path)[0])[0] + "/scripts"680 os.path.split(self.accomplishments_path)[0])[0] + "/scripts"
519 logging.info(681 log.msg(
520 "...setting scripts path to: " + self.scripts_path)682 "...setting scripts path to: " + self.scripts_path)
521 if config.get('config', 'trophypath'):683 if config.get('config', 'trophypath'):
522 logging.info(684 log.msg(
523 "...setting trophies path to: "685 "...setting trophies path to: "
524 + config.get('config', 'trophypath'))686 + config.get('config', 'trophypath'))
525 self.trophies_path = config.get('config', 'trophypath')687 self.trophies_path = config.get('config', 'trophypath')
@@ -529,84 +691,23 @@
529 self.has_verif = config.getboolean('config', 'has_verif')691 self.has_verif = config.getboolean('config', 'has_verif')
530 else:692 else:
531 accompath = os.path.join(homedir, "accomplishments")693 accompath = os.path.join(homedir, "accomplishments")
532 logging.info("Configuration file not found...creating it!")694 log.msg("Configuration file not found...creating it!")
533 print "Configuration file not found...creating it!"
534695
535 self.has_verif = False696 self.has_verif = False
536 self.accomplishments_path = accompath697 self.accomplishments_path = accompath
537 logging.info(698 log.msg(
538 "...setting accomplishments path to: "699 "...setting accomplishments path to: "
539 + self.accomplishments_path)700 + self.accomplishments_path)
540 self.trophies_path = os.path.join(self.dir_data, "trophies")701 self.trophies_path = os.path.join(self.dir_data, "trophies")
541 logging.info("...setting trophies path to: " + self.trophies_path)702 log.msg("...setting trophies path to: " + self.trophies_path)
542 self.scripts_path = os.path.join(accompath, "scripts")703 self.scripts_path = os.path.join(accompath, "scripts")
543 logging.info("...setting scripts path to: " + self.scripts_path)704 log.msg("...setting scripts path to: " + self.scripts_path)
544705
545 if not os.path.exists(self.trophies_path):706 if not os.path.exists(self.trophies_path):
546 os.makedirs(self.trophies_path)707 os.makedirs(self.trophies_path)
547708
548 self._writeConfigFile()709 self._writeConfigFile()
549710
550 @defer.inlineCallbacks
551 def registerTrophyDir(self, trophydir):
552 """
553 Creates the Ubuntu One share for the trophydir and offers it to the
554 server. Returns True if the folder was successfully shared, False if
555 not.
556 """
557 timeid = str(time.time())
558 logging.info("Registering Ubuntu One share directory: " + trophydir)
559
560 folder_list = yield self.sd.get_folders()
561 folder_is_synced = False
562 for folder in folder_list:
563 if folder["path"] == trophydir:
564 folder_is_synced = True
565 break
566 if not folder_is_synced:
567 logging.info(
568 "...the '%s' folder is not synced with the Matrix" % trophydir)
569 logging.info("...creating the share folder on Ubuntu One")
570 self.sd.create_folder(trophydir)
571
572 success_filter = lambda info: info["path"] == trophydir
573 info = yield self.sd.wait_for_signals(
574 signal_ok='FolderCreated', success_filter=success_filter)
575
576 self.sd.offer_share(
577 trophydir, MATRIX_USERNAME, LOCAL_USERNAME + " Trophies Folder"
578 + " (" + timeid + ")", "Modify")
579 logging.info(
580 "...share has been offered (" + trophydir + "" + ", "
581 + MATRIX_USERNAME + ", " + LOCAL_USERNAME + ")")
582 return
583
584 logging.info(
585 "...the '%s' folder is already synced... now checking whether "
586 "it's shared" % trophydir)
587 shared_list = yield self.sd.list_shared()
588 folder_is_shared = False
589 shared_to = []
590 for share in shared_list:
591 if share["path"] == trophydir:
592 logging.info("...the folder is already shared.")
593 folder_is_shared = True
594 shared_to.append("%s (%s)" % (
595 share["other_visible_name"], share["other_username"]))
596 if not folder_is_shared:
597 logging.info("...the '%s' folder is not shared" % trophydir)
598 self.sd.offer_share(
599 trophydir, MATRIX_USERNAME, LOCAL_USERNAME + " Trophies Folder"
600 + " (" + timeid + ")", "Modify")
601 logging.info("...share has been offered (" + trophydir + "" + ", "
602 + MATRIX_USERNAME + ", " + LOCAL_USERNAME + ")")
603 logging.info("...offered the share.")
604 return
605 else:
606 logging.info("The folder is shared, with: %s" % ", ".join(
607 shared_to))
608 return
609
610 def getAllExtraInformationRequired(self):711 def getAllExtraInformationRequired(self):
611 """712 """
612 Return a dictionary of all information required for the accomplishments713 Return a dictionary of all information required for the accomplishments
@@ -664,7 +765,7 @@
664 else:765 else:
665 getdepends = False766 getdepends = False
666767
667 logging.info("List all accomplishments and status")768 log.msg("List all accomplishments and status")
668 accomplishments_files = self._get_accomplishments_files_list()769 accomplishments_files = self._get_accomplishments_files_list()
669 things = {}770 things = {}
670 for accomplishment_file in accomplishments_files:771 for accomplishment_file in accomplishments_files:
@@ -748,7 +849,7 @@
748 return things.values()849 return things.values()
749850
750 def listAllAvailableAccomplishmentsWithScripts(self):851 def listAllAvailableAccomplishmentsWithScripts(self):
751 logging.info("List all accomplishments with scripts")852 log.msg("List all accomplishments with scripts")
752 available = [accom for accom in self.listAllAccomplishmentsAndStatus()853 available = [accom for accom in self.listAllAccomplishmentsAndStatus()
753 if not accom["accomplished"] and not accom["locked"]]854 if not accom["accomplished"] and not accom["locked"]]
754 withscripts = []855 withscripts = []
@@ -764,7 +865,7 @@
764 return withscripts865 return withscripts
765866
766 def listAccomplishmentInfo(self, accomplishment):867 def listAccomplishmentInfo(self, accomplishment):
767 logging.info("Getting accomplishment info for " + accomplishment)868 log.msg("Getting accomplishment info for " + accomplishment)
768 search = "/" + accomplishment + ".accomplishment"869 search = "/" + accomplishment + ".accomplishment"
769 files = self._get_accomplishments_files_list()870 files = self._get_accomplishments_files_list()
770 match = None871 match = None
@@ -780,119 +881,23 @@
780 data.append(dict(config._sections["accomplishment"]))881 data.append(dict(config._sections["accomplishment"]))
781 return data882 return data
782883
783 @defer.inlineCallbacks
784 def run_scripts_for_user(self, uid):
785 print "--- Starting Running Scripts ---"
786 logging.info("--- Starting Running Scripts ---")
787 timestart = time.time()
788 self.service.scriptrunner_start()
789
790 # Is the user currently logged in and running a gnome session?
791 username = pwd.getpwuid(uid).pw_name
792 try:
793 proc = subprocess.check_output(
794 ["pgrep", "-u", username, "gnome-session"]).strip()
795 except subprocess.CalledProcessError:
796 # user does not have gnome-session running or isn't logged in at
797 # all
798 logging.info("No gnome-session process for user %s" % username)
799 return
800 fp = open("/proc/%s/environ" % proc)
801 try:
802 envars = dict(
803 [line.split("=", 1) for line in fp.read().split("\0")
804 if line.strip()])
805 except IOError:
806 # user does not have gnome-session running or isn't logged in at
807 # all
808 logging.info("No gnome-session environment for user %s" % username)
809 return
810 fp.close()
811
812 os.seteuid(uid)
813
814 required_envars = ['DBUS_SESSION_BUS_ADDRESS']
815 env = dict([kv for kv in envars.items() if kv[0] in required_envars])
816 oldenviron = os.environ
817 os.environ.update(env)
818 # XXX this DBUS check happens in the middle of this method; it would be
819 # better if this check was done at a higher level, for instance, where
820 # this class is initiated: if the daemon isn't registered at the time
821 # of instantiation, simply abort then instead of making all the way
822 # here and then aborting. (Note that moving this check to that location
823 # will also eliminate an obvious circular import.)
824 if not dbusapi.daemon_is_registered():
825 return
826
827 accoms = self.listAllAvailableAccomplishmentsWithScripts()
828 totalscripts = len(accoms)
829 self.scriptrun_total = totalscripts
830 logging.info("Need to run (%d) scripts" % totalscripts)
831 print "Need to run (%d) scripts" % totalscripts
832
833 scriptcount = 1
834 for accom in accoms:
835 msg = "%s/%s: %s" % (scriptcount, totalscripts, accom["_script"])
836 print msg
837 logging.info(msg)
838 exitcode = yield self.run_a_subprocess([accom["_script"]])
839 if exitcode == 0:
840 self.scriptrun_results.append(
841 str(accom["application"]) + "/"
842 + str(accom["accomplishment"]))
843 self.accomplish(accom["application"], accom["accomplishment"])
844 print "...Accomplished"
845 logging.info("...Accomplished")
846 elif exitcode == 1:
847 self.scriptrun_results.append(None)
848 print "...Not Accomplished"
849 logging.info("...Not Accomplished")
850 elif exitcode == 2:
851 self.scriptrun_results.append(None)
852 print "...Error"
853 logging.info("....Error")
854 else:
855 self.scriptrun_results.append(None)
856 print "...Other error code."
857 logging.info("...Other error code")
858 scriptcount = scriptcount + 1
859
860 os.environ = oldenviron
861
862 timeend = time.time()
863 timefinal = round((timeend - timestart), 2)
864
865 print "--- Completed Running Scripts in %.2f seconds ---" % timefinal
866 logging.info(
867 "--- Completed Running Scripts in %.2f seconds---" % timefinal)
868 self.service.scriptrunner_finish()
869
870 def run_a_subprocess(self, command):
871 logging.info("Running subprocess command: " + str(command))
872 pprotocol = SubprocessReturnCodeProtocol()
873 reactor.spawnProcess(pprotocol, command[0], command, env=os.environ)
874 return pprotocol.returnCodeDeferred
875
876 def run_scripts_for_all_active_users(self):884 def run_scripts_for_all_active_users(self):
877 for uid in [x.pw_uid for x in pwd.getpwall()885 for uid in [x.pw_uid for x in pwd.getpwall()
878 if x.pw_dir.startswith('/home/') and x.pw_shell != '/bin/false']:886 if x.pw_dir.startswith('/home/') and x.pw_shell != '/bin/false']:
879 os.seteuid(0)887 os.seteuid(0)
880 self.run_scripts_for_user(uid)888 self.asyncapi.run_scripts_for_user(uid)
881889
882 def run_scripts(self, run_by_client):890 def run_scripts(self, run_by_client):
883 uid = os.getuid()891 uid = os.getuid()
884 if uid == 0:892 if uid == 0:
885 logging.info("Run scripts for all active users")893 log.msg("Run scripts for all active users")
886 self.run_scripts_for_all_active_users()894 self.run_scripts_for_all_active_users()
887 else:895 else:
888 logging.info("Run scripts for user")896 log.msg("Run scripts for user")
889 self.run_scripts_for_user(uid)897 self.asyncapi.run_scripts_for_user(uid)
890
891 if run_by_client is False:
892 reactor.callLater(SCRIPT_DELAY, self.run_scripts, False)
893898
894 def createExtraInformationFile(self, app, item, data):899 def createExtraInformationFile(self, app, item, data):
895 logging.info(900 log.msg(
896 "Creating Extra Information file: %s, %s, %s", app, item, data)901 "Creating Extra Information file: %s, %s, %s", app, item, data)
897 extrainfodir = os.path.join(self.trophies_path, ".extrainformation/")902 extrainfodir = os.path.join(self.trophies_path, ".extrainformation/")
898903
@@ -906,7 +911,6 @@
906 f.write(data)911 f.write(data)
907 f.close()912 f.close()
908913
909
910 def getExtraInformation(self, app, item):914 def getExtraInformation(self, app, item):
911 extrainfopath = os.path.join(self.trophies_path, ".extrainformation/")915 extrainfopath = os.path.join(self.trophies_path, ".extrainformation/")
912 authfile = os.path.join(extrainfopath, item)916 authfile = os.path.join(extrainfopath, item)
@@ -918,10 +922,3 @@
918 #print "No data."922 #print "No data."
919 final = [{item : False}]923 final = [{item : False}]
920 return final924 return final
921
922
923 # XXX once the other reference to dbusapi is removed from this file, this
924 # will be the last one. It doesn't really belong here... we can set a whole
925 # slew of dbus api calls on this object when it is instantiated, thus
926 # alleviating us from the burden of a hack like this below
927 trophy_received = dbusapi.DBUSSignals.trophy_received
928925
=== modified file 'accomplishments/daemon/app.py'
--- accomplishments/daemon/app.py 2012-03-18 16:56:29 +0000
+++ accomplishments/daemon/app.py 2012-03-20 03:24:21 +0000
@@ -1,13 +1,15 @@
1from optparse import OptionParser1from optparse import OptionParser
22
3from twisted.internet import glib2reactor3from twisted.application.internet import TimerService
4glib2reactor.install()4from twisted.application.service import Application
5from twisted.internet import reactor
65
7from accomplishments.daemon import dbusapi6from accomplishments.daemon import dbusapi
87from accomplishments.daemon import service
98
10if __name__ == "__main__":9
10# XXX these won't work with twistd; we need to write a twistd plugin to support
11# additional command line options.
12def parse_options():
11 parser = OptionParser()13 parser = OptionParser()
12 parser.set_defaults(suppress_notifications=False)14 parser.set_defaults(suppress_notifications=False)
13 parser.add_option("--trophies-path", dest="trophies_path", default=None)15 parser.add_option("--trophies-path", dest="trophies_path", default=None)
@@ -16,8 +18,31 @@
16 parser.add_option("--scripts-path", dest="scripts_path", default=None)18 parser.add_option("--scripts-path", dest="scripts_path", default=None)
17 parser.add_option("--suppress-notifications", action="store_true",19 parser.add_option("--suppress-notifications", action="store_true",
18 dest="suppress_notifications")20 dest="suppress_notifications")
19 options, args = parser.parse_args()21 return parser.parse_args()
2022
21 service = dbusapi.AccomplishmentsService(23
22 show_notifications=not options.suppress_notifications)24def applicationFactory(app_name="", bus_name="", main_loop=None,
23 reactor.run()25 session_bus=None, object_path="/", update_interval=3600,
26 gpg_key=""):
27 # create the application object
28 application = Application(app_name)
29 # create the top-level service object that will contain all others; it will
30 # not shutdown until all child services have been terminated
31 top_level_service = service.AccomplishmentsDaemonService(gpg_key)
32 top_level_service.setServiceParent(application)
33 # create the service that all DBus services will rely upon (this parent
34 # service will wait until child DBus services are shutdown before it shuts
35 # down
36 dbus_service = service.DBusService(main_loop, session_bus)
37 dbus_service.setServiceParent(top_level_service)
38 # create a child dbus serivce
39 dbus_export_service = dbusapi.AccomplishmentsDBusService(
40 bus_name, session_bus, object_path=object_path,
41 show_notifications=False)
42 dbus_export_service.setServiceParent(dbus_service)
43 # create a service that will run the scripts at a regular interval
44 timer_service = service.ScriptRunnerService(
45 update_interval, dbus_export_service.api)
46 timer_service.setServiceParent(top_level_service)
47
48 return application
2449
=== modified file 'accomplishments/daemon/dbusapi.py'
--- accomplishments/daemon/dbusapi.py 2012-03-18 16:56:29 +0000
+++ accomplishments/daemon/dbusapi.py 2012-03-20 03:24:21 +0000
@@ -3,11 +3,10 @@
3# GObject-introspection-capable C thing so anyone can use it. But someone else3# GObject-introspection-capable C thing so anyone can use it. But someone else
4# needs to write that because I'm crap at Vala.4# needs to write that because I'm crap at Vala.
5import dbus5import dbus
6import dbus.service6
7from dbus.mainloop.glib import DBusGMainLoop7from twisted.python import log
88
99from accomplishments.daemon import service
10DBusGMainLoop(set_as_default=True)
1110
1211
13def daemon_is_registered():12def daemon_is_registered():
@@ -18,7 +17,7 @@
18 "org.ubuntu.accomplishments", "/")17 "org.ubuntu.accomplishments", "/")
19 return True18 return True
20 except dbus.exceptions.DBusException:19 except dbus.exceptions.DBusException:
21 logging.info(20 log.msg(
22 "User %s does not have the accomplishments daemon "21 "User %s does not have the accomplishments daemon "
23 "available" % username)22 "available" % username)
24 return False23 return False
@@ -26,6 +25,17 @@
2625
27# XXX as a function, this needs to be renamed to a lower-case function name and26# XXX as a function, this needs to be renamed to a lower-case function name and
28# not use the class naming convention of upper-case.27# not use the class naming convention of upper-case.
28#
29# Hrm, on second thought... this is:
30# * a singleton object (SessionBus)
31# * obtaining an object indicated by string constants
32# * and then doing a lookup on these
33# In other words, nothing changes ;-)
34#
35# As such, there's no need for this to be a function; instead, we can set our
36# own module-level singleton, perhaps as part of the AccomplishmentsDBusService
37# set up, since that object has access to all the configuration used here (bus
38# name and bus path).
29def Accomplishments():39def Accomplishments():
30 """40 """
31 """41 """
@@ -33,23 +43,14 @@
33 return dbus.Interface(obj, "org.ubuntu.accomplishments")43 return dbus.Interface(obj, "org.ubuntu.accomplishments")
3444
3545
36class DBUSSignals(object):46class AccomplishmentsDBusService(service.DBusExportService):
37 """47 """
38 """48 """
39 @staticmethod49 def __init__(self, bus_name, session_bus, object_path="/",
40 @dbus.service.signal(dbus_interface='org.ubuntu.accomplishments')50 show_notifications=True):
41 def trophy_received(self, trophy):51 super(AccomplishmentsDBusService, self).__init__(bus_name, session_bus)
42 t = "mytrophy"52 bus_name = dbus.service.BusName(bus_name, bus=session_bus)
43 return t53 dbus.service.Object.__init__(self, bus_name, object_path)
44
45
46class AccomplishmentsService(dbus.service.Object):
47 """
48 """
49 def __init__(self, show_notifications=True):
50 bus_name = dbus.service.BusName(
51 'org.ubuntu.accomplishments', bus=dbus.SessionBus())
52 dbus.service.Object.__init__(self, bus_name, '/')
53 self.show_notifications = show_notifications54 self.show_notifications = show_notifications
54 # XXX until all the imports are cleaned up and the code is organized55 # XXX until all the imports are cleaned up and the code is organized
55 # properly, we're doing the import here (to avoid circular imports).56 # properly, we're doing the import here (to avoid circular imports).
@@ -57,76 +58,83 @@
57 # this is not a subclass of dbus.service.Object *and* Accomplishments58 # this is not a subclass of dbus.service.Object *and* Accomplishments
58 # because doing that confuses everything, so we create our own59 # because doing that confuses everything, so we create our own
59 # private Accomplishments object and use it.60 # private Accomplishments object and use it.
60 self.ad = api.Accomplishments(self, self.show_notifications)61 self.api = api.Accomplishments(self, self.show_notifications)
6162
62 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',63 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
63 in_signature="", out_signature="aa{sv}")64 in_signature="", out_signature="aa{sv}")
64 def listAllAccomplishments(self):65 def listAllAccomplishments(self):
65 return self.ad.listAllAccomplishments()66 return self.api.listAllAccomplishments()
6667
67 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',68 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
68 in_signature="", out_signature="aa{sv}")69 in_signature="", out_signature="aa{sv}")
69 def listAllAccomplishmentsAndStatus(self):70 def listAllAccomplishmentsAndStatus(self):
70 return self.ad.listAllAccomplishmentsAndStatus()71 return self.api.listAllAccomplishmentsAndStatus()
7172
72 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',73 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
73 in_signature="", out_signature="aa{sv}")74 in_signature="", out_signature="aa{sv}")
74 def listAllAvailableAccomplishmentsWithScripts(self):75 def listAllAvailableAccomplishmentsWithScripts(self):
75 return self.ad.listAllAvailableAccomplishmentsWithScripts()76 return self.api.listAllAvailableAccomplishmentsWithScripts()
7677
77 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',78 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
78 in_signature="", out_signature="aa{sv}")79 in_signature="", out_signature="aa{sv}")
79 def getAllExtraInformationRequired(self):80 def getAllExtraInformationRequired(self):
80 return self.ad.getAllExtraInformationRequired()81 return self.api.getAllExtraInformationRequired()
8182
82 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',83 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
83 in_signature="", out_signature="aa{sv}")84 in_signature="", out_signature="aa{sv}")
84 def listAccomplishmentInfo(self, accomplishment):85 def listAccomplishmentInfo(self, accomplishment):
85 return self.ad.listAccomplishmentInfo(accomplishment)86 return self.api.listAccomplishmentInfo(accomplishment)
8687
87 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',88 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
88 in_signature="b", out_signature="")89 in_signature="b", out_signature="")
89 def run_scripts(self, run_by_client):90 def run_scripts(self, run_by_client):
90 return self.ad.run_scripts(run_by_client)91 return self.api.run_scripts(run_by_client)
9192
92 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',93 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
93 in_signature="", out_signature="aa{sv}")94 in_signature="", out_signature="aa{sv}")
94 def getExtraInformation(self, app, info):95 def getExtraInformation(self, app, info):
95 return self.ad.getExtraInformation(app, info)96 return self.api.getExtraInformation(app, info)
9697
97 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',98 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
98 in_signature="", out_signature="")99 in_signature="", out_signature="")
99 def createExtraInformationFile(self, app, item, data):100 def createExtraInformationFile(self, app, item, data):
100 return self.ad.createExtraInformationFile(app, item, data)101 return self.api.createExtraInformationFile(app, item, data)
101102
102 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',103 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
103 in_signature="ss", out_signature="")104 in_signature="ss", out_signature="")
104 def accomplish(self, app, accomplishment_name):105 def accomplish(self, app, accomplishment_name):
105 trophy = self.ad.accomplish(app, accomplishment_name)106 trophy = self.api.accomplish(app, accomplishment_name)
106107
107 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',108 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
108 in_signature="s", out_signature="b")109 in_signature="s", out_signature="b")
109 def registerTrophyDir(self, trophydir):110 def register_trophy_dir(self, trophydir):
110 return self.ad.registerTrophyDir(trophydir)111 return self.api.async.register_trophy_dir(trophydir)
111112
112 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',113 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
113 in_signature="vv", out_signature="v")114 in_signature="vv", out_signature="v")
114 def getConfigValue(self, section, item):115 def getConfigValue(self, section, item):
115 return self.ad.getConfigValue(section, item)116 return self.api.getConfigValue(section, item)
116117
117 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',118 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
118 in_signature="vvv", out_signature="")119 in_signature="vvv", out_signature="")
119 def setConfigValue(self, section, item, value):120 def setConfigValue(self, section, item, value):
120 return self.ad.setConfigValue(section, item, value)121 return self.api.setConfigValue(section, item, value)
121122
122 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',123 @dbus.service.method(dbus_interface='org.ubuntu.accomplishments',
123 in_signature="", out_signature="b")124 in_signature="", out_signature="b")
124 def verifyU1Account(self):125 def verifyU1Account(self):
125 return self.ad.verifyU1Account()126 return self.api.verifyU1Account()
126127
128 # XXX this looks like an unintentional duplicate of the "other"
129 # trophy_received... I've moved them here together so that someone in the
130 # know (Jono?) can clarify and remove the one that's not needed
127 @dbus.service.signal(dbus_interface='org.ubuntu.accomplishments')131 @dbus.service.signal(dbus_interface='org.ubuntu.accomplishments')
128 def trophy_received(self, trophy):132 def trophy_received(self, trophy):
129 pass133 pass
134 @dbus.service.signal(dbus_interface='org.ubuntu.accomplishments')
135 def trophy_received(trophy):
136 t = "mytrophy"
137 return t
130138
131 @dbus.service.signal(dbus_interface='org.ubuntu.accomplishments')139 @dbus.service.signal(dbus_interface='org.ubuntu.accomplishments')
132 def scriptrunner_start(self):140 def scriptrunner_start(self):
133141
=== added file 'accomplishments/daemon/service.py'
--- accomplishments/daemon/service.py 1970-01-01 00:00:00 +0000
+++ accomplishments/daemon/service.py 2012-03-20 03:24:21 +0000
@@ -0,0 +1,80 @@
1from twisted.application.internet import TimerService
2from twisted.application.service import MultiService
3from twisted.application.service import Service
4from twisted.python import log
5
6import dbus.service
7
8from accomplishments import util
9
10
11class AccomplishmentsDaemonService(MultiService):
12 """
13 The top-level service that all other services should set as their parent.
14 """
15 def __init__(self, gpg_pub_key):
16 MultiService.__init__(self)
17 self.gpg_key = gpg_pub_key
18
19 def startService(self):
20 log.msg("Starting up accomplishments daemon ...")
21 util.import_gpg_key(self.gpg_key)
22 MultiService.startService(self)
23
24 def stopService(self):
25 log.msg("Shutting down accomplishments daemon ...")
26 return MultiService.stopService(self)
27
28
29class DBusService(MultiService):
30 """
31 A Twisted application service that tracks the DBus mainloop and the DBus
32 session bus.
33 """
34 def __init__(self, main_loop, session_bus):
35 MultiService.__init__(self)
36 self.main_loop = main_loop
37 self.session_bus = session_bus
38
39 def startService(self):
40 log.msg("Starting up DBus service ...")
41 return MultiService.startService(self)
42
43 def stopService(self):
44 log.msg("Shutting down DBus service ...")
45 return MultiService.stopService(self)
46
47
48class DBusExportService(Service, dbus.service.Object):
49 """
50 A base class that is both a Twisted application service as well as a means
51 for exporting custom objects across a given bus.
52 """
53 def __init__(self, bus_name, session_bus):
54 self.bus_name = bus_name
55 self.session_bus = session_bus
56
57 def startService(self):
58 log.msg("Starting up API exporter service ...")
59 return Service.startService(self)
60
61 def stopService(self):
62 log.msg("Shutting down API exporter service ...")
63 return Service.stopService(self)
64
65
66class ScriptRunnerService(TimerService):
67 """
68 A simple wrapper for the TimerService that runs the scripts at the given
69 intertal.
70 """
71 def __init__(self, interval, api):
72 TimerService.__init__(self, interval, api.run_scripts, False)
73
74 def startService(self):
75 log.msg("Starting up script runner service ...")
76 return TimerService.startService(self)
77
78 def stopService(self):
79 log.msg("Shutting down script runner service ...")
80 return TimerService.stopService(self)
081
=== modified file 'accomplishments/gui/TrophyinfoWindow.py'
--- accomplishments/gui/TrophyinfoWindow.py 2012-03-12 16:53:31 +0000
+++ accomplishments/gui/TrophyinfoWindow.py 2012-03-20 03:24:21 +0000
@@ -233,7 +233,7 @@
233 self.has_verif = True233 self.has_verif = True
234 self.libaccom.setConfigValue("config", "has_verif", True)234 self.libaccom.setConfigValue("config", "has_verif", True)
235235
236 res = self.libaccom.registerTrophyDir(trophydir)236 res = self.libaccom.asyncapi.register_trophy_dir(trophydir)
237237
238 if res == 1:238 if res == 1:
239 self.u1_button.set_label("Successfully shared. Click here to continue...")239 self.u1_button.set_label("Successfully shared. Click here to continue...")
240240
=== modified file 'accomplishments/util/__init__.py'
--- accomplishments/util/__init__.py 2012-03-05 03:49:28 +0000
+++ accomplishments/util/__init__.py 2012-03-20 03:24:21 +0000
@@ -5,6 +5,11 @@
5import gettext5import gettext
6from gettext import gettext as _6from gettext import gettext as _
77
8from twisted.internet import defer
9from twisted.internet import protocol
10from twisted.internet import reactor
11from twisted.python import log
12
8import accomplishments13import accomplishments
9from accomplishments import config14from accomplishments import config
10from accomplishments import exceptions15from accomplishments import exceptions
@@ -43,7 +48,7 @@
43 logger.setLevel(logging.DEBUG)48 logger.setLevel(logging.DEBUG)
44 logger.debug('logging enabled')49 logger.debug('logging enabled')
45 if opts.verbose > 1:50 if opts.verbose > 1:
46 lib_logger.setLevel(logging.DEBUG)51 logger.setLevel(logging.DEBUG)
4752
4853
49def get_data_path():54def get_data_path():
@@ -86,3 +91,34 @@
86 help=_("Clear your trophies collection"))91 help=_("Clear your trophies collection"))
87 (options, args) = parser.parse_args()92 (options, args) = parser.parse_args()
88 return options93 return options
94
95
96class SubprocessReturnCodeProtocol(protocol.ProcessProtocol):
97 """
98 """
99 def __init__(self, command=""):
100 self.command = command
101
102 def connectionMade(self):
103 self.returnCodeDeferred = defer.Deferred()
104
105 def processEnded(self, reason):
106 self.returnCodeDeferred.callback(reason.value.exitCode)
107
108 def outReceived(self, data):
109 log.msg("Got process results: %s" % data)
110
111 def errReceived(self, data):
112 log.err("Got non-zero exit code for process: %s" % (
113 " ".join(self.command),))
114 log.msg(data)
115
116
117def import_gpg_key(pub_key):
118 """
119 """
120 cmd = ["gpg", "--import", pub_key]
121 gpg = SubprocessReturnCodeProtocol(cmd)
122 gpg.deferred = defer.Deferred()
123 process = reactor.spawnProcess(gpg, cmd[0], cmd, env=None)
124 return gpg.deferred
89125
=== renamed file 'bin/rundaemon.sh' => 'bin/daemon'
--- bin/rundaemon.sh 2012-03-05 03:51:41 +0000
+++ bin/daemon 2012-03-20 03:24:21 +0000
@@ -1,15 +1,28 @@
1#!/bin/bash1#!/usr/bin/twistd
22"""
3#D="../../"3Run the Accomplishments daemon!
4export PYTHONPATH=$PYTHONPATH:.4"""
55import sys
6echo Importing the validation key...6
7gpg --import ./data/daemon/validation-key.pub7from twisted.internet import glib2reactor
88glib2reactor.install()
9echo Starting the accomplishments daemon...9
10echo "(this would be done by D-Bus activation in the real world)"10import dbus
1111from dbus.mainloop.glib import DBusGMainLoop
12python ./accomplishments/daemon/app.py12
13DAEMON=$!13from accomplishments.daemon import app
1414
1515
16# PYTHONPATH mangling
17sys.path.insert(0, ".")
18
19dbus_loop = DBusGMainLoop(set_as_default=True)
20application = app.applicationFactory(
21 app_name="Ubuntu Accomplishments",
22 bus_name="org.ubuntu.accomplishments",
23 main_loop=dbus_loop,
24 session_bus=dbus.SessionBus(mainloop=dbus_loop),
25 object_path="/",
26 # Let's execute the timer service every 15 minutes
27 update_interval=15 * 60,
28 gpg_key="./data/daemon/validation-key.pub")

Subscribers

People subscribed via source and target branches