Merge lp:~widelands-dev/widelands-website/encyclopedia into lp:widelands-website

Proposed by GunChleoc
Status: Merged
Merged at revision: 406
Proposed branch: lp:~widelands-dev/widelands-website/encyclopedia
Merge into: lp:widelands-website
Diff against target: 1434 lines (+534/-409)
10 files modified
templates/wlhelp/index.html (+0/-2)
templates/wlhelp/inlines/display_buildings.html (+1/-1)
widelandslib/make_flow_diagram.py (+22/-17)
widelandslib/tribe.py (+79/-89)
wlhelp/management/commands/update_help.py (+227/-144)
wlhelp/management/commands/update_help_pdf.py (+23/-17)
wlhelp/models.py (+91/-61)
wlhelp/tests.py (+5/-4)
wlhelp/urls.py (+22/-15)
wlhelp/views.py (+64/-59)
To merge this branch: bzr merge lp:~widelands-dev/widelands-website/encyclopedia
Reviewer Review Type Date Requested Status
kaputtnik (community) in widelands code Needs Fixing
GunChleoc Needs Resubmitting
SirVer Approve
Review via email: mp+287410@code.launchpad.net

Commit message

Update the encyclopedia from JSON files.

Description of the change

Update the encyclopedia from JSON files.

This will also call the new executable first to make sure that the JSON files have been written. Also checks that the files are valid JSON.

Also deletes the models in the database before regenerating, so we will get rid of the old duplicates.

There shouldn't be any other functionality changes.

Related Widelands branch:

https://code.launchpad.net/~widelands-dev/widelands/map_object_info

To post a comment you must log in.
422. By GunChleoc

Code cleanup.

Revision history for this message
SirVer (sirver) wrote :

must not be merged before the corresponding WL branch is deployed on the website.

lgtm to me in general, though I did not test it. What is the plan for generating the JSON files? It seems you expect them to be dumped into the JSON directory in the repo - do you want to check those files into the repo or should they be regenerated on the server?

review: Approve
Revision history for this message
GunChleoc (gunchleoc) wrote :

My idea is to generate them on the server - I have added a call to the executable in update_help.py. It will also run a python script that checks for basic JSON validity. My idea is that they aren't part of the Widelands installation as such and that they shouldn't be packaged, so I prefer to generate them this way.

We should test this on the alpha server before deploying it on the live server.

Revision history for this message
SirVer (sirver) wrote :

I strongly advise packaging the binary into the .deb then - it will be deployed on the server via the PPA which is the least hassle. And maybe a user enjoys playing with it too.

> On 29.02.2016, at 10:36, GunChleoc <email address hidden> wrote:
>
> My idea is to generate them on the server - I have added a call to the executable in update_help.py. It will also run a python script that checks for basic JSON validity. My idea is that they aren't part of the Widelands installation as such and that they shouldn't be packaged, so I prefer to generate them this way.
>
> We should test this on the alpha server before deploying it on the live server.
> --
> https://code.launchpad.net/~widelands-dev/widelands-website/encyclopedia/+merge/287410
> You are reviewing the proposed merge of lp:~widelands-dev/widelands-website/encyclopedia into lp:widelands-website.

Revision history for this message
GunChleoc (gunchleoc) wrote :

It is already part of the .deb - I just tested with sudo make install.

However, when I try to change the Python call to

    subprocess.check_call([os.path.normpath('wl_map_object_info')])

it doesn't work.

Revision history for this message
SirVer (sirver) wrote :

On the server/.deb file it is installed into /usr/share/games which is not usual in the search path. You might need to use the full path to the binary or change the PATH variable.

> Am 02.03.2016 um 15:49 schrieb GunChleoc <email address hidden>:
>
> It is already part of the .deb - I just tested with sudo make install.
>
> However, when I try to change the Python call to
>
> subprocess.check_call([os.path.normpath('wl_map_object_info')])
>
> it doesn't work.
> --
> https://code.launchpad.net/~widelands-dev/widelands-website/encyclopedia/+merge/287410
> You are reviewing the proposed merge of lp:~widelands-dev/widelands-website/encyclopedia into lp:widelands-website.

Revision history for this message
kaputtnik (franku) wrote :

Works :-)

Maybe we could adjust media/css/help.css to fit the table style with the one from the wiki? Currently the table headers have a really small font size and does not not show the wooden background. You could load the wiki.css before help.css and adjust only disturbing things then in help.css.

Open the link for gif in tribe_detail do actually open a new tab/window. I believe this is a bad behavior. Same goes for the pdf-link but here the pdf is downloaded and is shown with my pdf-viewer.

Some texts are missing (ore's) but i believe this depends on the helptexts in the init.lua.

Please take a look on the diff comment from SirVer (if not done until now).

Nice work :-)

review: Approve (testing)
Revision history for this message
GunChleoc (gunchleoc) wrote :

Thanks for pointing out the diff comment, I had missed that. We won't only get 404s when deleting the media files, but also when deleting the Django models. How about we add a commandline switch to the update script, so we can decide whether we need to clean house or not? We won't need to do this often, because we're pretty much done renaming tribe entities.

I managed to run Widelands from an installation now, but we will need to add an argument to the executable now to choose the path for the JSON files, because we won't want to run this with sudo.

Revision history for this message
kaputtnik (franku) wrote :

> I managed to run Widelands from an installation now, but we will need to add an argument to the executable now to choose the path for the JSON files, because we won't want to run this with sudo.

Could you please explain a bit more?

Because widelands is installed on the server, it should work ootb? Normally a global installed program could be run by every user on the machine, so sudo isn't needed.

Maybe we could add a button on the admin page for updating the help? Oh i see that wlhelp haven't a admin site defined at all ...

Revision history for this message
GunChleoc (gunchleoc) wrote :

If we install Widelands from the PPA, it will end up in /usr/local/games. The executable then tries to write the JSON files to /usr/local/games/data/map_object_info. Permission is denied for creating the directory. It's a security measure - a bit like Windows getting prissy when you try to mess with C:\Program Files and you have to confirm everything.

Revision history for this message
kaputtnik (franku) wrote :

Ah, now i understand... the update process is a bit uncomfortable anyway:

1. call '[sudo] wl_map_object_info' to generate updated json files
2. call ./manage.py update_help to update the database

The json files are only needed by the website, so why not call wl_map_object_info from update_help.py and storing the files somewhere beneath MEDIA_ROOT? Something similar like we have with uploading a map.

Now my ssh access on the server is broken :-(

Revision history for this message
GunChleoc (gunchleoc) wrote :

> The json files are only needed by the website, so why not call
> wl_map_object_info from update_help.py

I am already doing that.

> and storing the files somewhere beneath
> MEDIA_ROOT? Something similar like we have with uploading a map.

That's the plan now :)

Revision history for this message
kaputtnik (franku) wrote :

> I am already doing that.

Oh i overlooked that, sorry.

Revision history for this message
GunChleoc (gunchleoc) wrote :

The C++ code has been updated now to specify the output dir on the command line. I can't get the website code to run after my recent system upgrade though, so I don't know how to finish up this branch :(

Revision history for this message
kaputtnik (franku) wrote :

Have you tried the wl_django1_8 branch? Even if some things won't work with this branch (mainly the third party apps) it should work for testing your changes. We could also try to submit my latest wlwebsite installation as a zipfile from me to you and you could try to use it. But i am not sure if this will work.

Hopefully i get ready with wl_django1_8 in a few weeks... at least for testing on the alpha site. Adapting the third party apps to django 1.8 and including them as wl-apps isn't much work, (i believe).

423. By GunChleoc

Fixed paths to take the media dir.

Revision history for this message
GunChleoc (gunchleoc) wrote :

> Have you tried the wl_django1_8 branch?

That was a great idea - should be ready to go now :)

review: Needs Resubmitting
Revision history for this message
kaputtnik (franku) wrote :

I have merged this branch into the actual website code and ran ./manage.py update_help and update_help_pdf

Both works fine :-) I didn't checked for correct output but i think it looks good. One thing i noticed: Military buildings does not show how many soldiers they could "store". But i didn't know how this was handled before.

Deploying this on the server needs some adjustments to file rights for WIDELANDS_SVN_DIR, because SirVer is the owner and some bzr actions won't work then. I have to adjust them in the same way we (SirVer and i ) did for the website folder/files.

review: Approve (testing)
Revision history for this message
GunChleoc (gunchleoc) wrote :

> One thing i noticed: Military buildings does not show how many soldiers
> they could "store". But i didn't know how this was handled before.

My version says "Keeps X Soldiers" for militarysites, both the current website and my branch.

Revision history for this message
kaputtnik (franku) wrote :

Ah, ok, i just looked in the tables in column "Stores" ... but this column meant only wares, not workers.

Some nits:
1. Please check for correct indentation (python prefers spaces for indentation). The file update_wlhelp.py has a mix of tabs and spaces. See also https://www.python.org/dev/peps/pep-0008/ You could just ran pyformat over the file(s)
2. The paths for images in the database contains three slashes after "/wlmedia", f.e.such a string is stored:
"/wlmedia///wlhelp/img/empire/empire_sentry/menu.png"

If you use database sqlite you could check the contents with:
sqlite3 dev.db #runs sqlite database console and open dev.db as database
select displayname, image_url from wlhelp_building where tribe_id=2; #returns all rows with tribe_id=2 (2=empire, 1=barbarians, 3=atlanteans) and shows the content of columns "displayname" and "image_url" from table "wlhelp_building"

You could use the second sql query also for mysql driven databases.

review: Needs Fixing (image_url)
424. By GunChleoc

Ran pyformat.

425. By GunChleoc

Run normpath over more paths.

426. By GunChleoc

Merged trunk.

Revision history for this message
GunChleoc (gunchleoc) wrote :

Should be all fixed now.

Revision history for this message
kaputtnik (franku) wrote :

Widelands is updated on the server and a cronjob was fixed to get the WIDELANDS_SVN_DIR updated (many thanks to janus for help).

I have installed this on the alpha site and get an error when trying to update the help:
-----------------------------------
$ ./manage.py update_help
JSON files will be written to: /var/www/django_projects/alpha/wlwebsite/code/widelands/media/map_object_info
selected language: en
No corresponding locale found - trying to set it via LANGUAGE=en, LANG=en
Exception: [/build/widelands-x6lppq/widelands-18-ppa0-bzr8009/src/website/map_object_info.cc:56] Unable to initialize SDL: Failed to connect to the Mir Server.
SoundHandler closing times 0, freq -1216634769, format 46873, chan -1222176768
Error: Unable to execute 'wl_map_object_info' for generating the JSON files.

Reading JSON files in: /var/www/django_projects/alpha/wlwebsite/code/widelands/media/map_object_info

All JSON files are OK.
-------------------------------

Seems that there is somewhere a dependency for an x-server... (Mir). Uploading maps works as usual, even it depends also on the graphics (wl_map_info). No json files where written. Any ideas?

Revision history for this message
Klaus Halfmann (klaus-halfmann) wrote :

kaputtnik: waht you need is a, so called, headless mode.

Either SDL can provide this (No idea how o do this, thiugh).
Or you can set um some X-Server thats works on some local
Bitmaps, which may be sufficent for your case.

Perhpas it is not needed so set up any graphics fo this case,
but the code forgot about that?

Just my 2cents of Knowledge....

Revision history for this message
kaputtnik (franku) wrote :

Because the culprit is in widelands wl_map_object_info and not on the website code, i leave this branch merged. But i do not deploy it on the productive website until wl_map_object_info is fixed.

Klaus, the map_info uses also an initialized SDL to get some informations of a map when someone wants to upload a map. The code looks similar to me, so i don't know why map_info works whereas map_object_info doesn't. And i am not as familiar with c++ than GunChleoc (or others) is/are.

The alpha site is stopped for now.

review: Needs Fixing (in widelands code)
Revision history for this message
GunChleoc (gunchleoc) wrote :

The only thing that is different is the call to i18n::set_locale("en"); in line 53 - could you try removing that line to see if it works? And if it does, move that line down?

Revision history for this message
kaputtnik (franku) wrote :

I don't believe that set_locale() is the culprit. I think when uploading a map the SDL is initialized through the browser and when trying to do it manually on the server it fails. This happens also with wl_map_info when trying to use it in console on the server ( i uploaded the map "Crossing the Horizon.wmf" for the test):

$ wl_map_info Crossing\ the\ Horizon.wmf
Exception: [/build/widelands-x6lppq/widelands-18-ppa0-bzr8009/src/website/map_info.cc:47] Unable to initialize SDL: Failed to connect to the Mir Server.

Remember: When i upload a map over the website, all is fine.

Either we find another way to parse the widelands configs, or we create a view for updating the encyclopedia so that the SDL is initialized through the browser. I would prefer the first solution... or we create an SDL independent way for widelands website related stuff (parsing maps and configs).

It would be interesting if wl_zocker could upload a map, because of the bugs which affects him. And to verify my assumption.

Revision history for this message
GunChleoc (gunchleoc) wrote :

OK, I get it now - I misunderstood and thought that wl_map_info was working.

I have no clue how to initiaize SDL on the server then. The easiest solution would be to ship the JSON files with the source code, like we do for the developers. It's a bit weird though to ship files with the game that the game itself doesn't use at all.

The only other idea that I can come up with right now is to find a way to load an egbase without graphics.

Revision history for this message
kaputtnik (franku) wrote :

It's a bit difficult to explain... wl_map_info is working on the server. It is used to get some informations when uploading a map and to extract the picture from the mapfile. Everytime a user uploads a map wl_map_info is executed.

The difference to wl_map_object_info is, that wl_map_info is startet in the context of the browser(someone has clicked "Upload map" in the webinterface). Executing it on the servers console, it fails to execute.

So i think that the browser context makes the difference. But i am not 100% sure. Just tested to upload a map with a textbrowser (elinks) and that succeeded.

Revision history for this message
GunChleoc (gunchleoc) wrote :

The browser context and the i18n:: call are the only differences in C++ until the point that the error occurs.

Maybe a Django admin module would do the trick?

Revision history for this message
SirVer (sirver) wrote :

Just a short drive by to explain how graphics on the server works: We run Xvfb, a virtual framebuffer that fakes a X11 environment at 800x600 at 32bit resolution that supports opengl through mesa.

This server runs on ":1" on the server, so running

export DISPLAY=":1"

and then running any of the widelands binaries that need graphics will work.

Revelant files:

/etc/init/xvfb.conf: starts the framebuffer
/etc/init/wlwebsite.conf: contains the lines
# Use Xvfb for displaying anything.
export DISPLAY=":1"

before starting anything else, so that the uploading can use wl_map_info.

Revision history for this message
kaputtnik (franku) wrote :

Thanks SirVer for explanation :-) So my thoughts where half correct :-D

> Maybe a Django admin module would do the trick?

Thats what i meant with "or we create a view for updating the encyclopedia so that the SDL is initialized through the browser." Now we know that the website uses the server on DISPLAY=":1" so a call of map_object_info from within the website code would work.

I look into that.

Revision history for this message
GunChleoc (gunchleoc) wrote :

Sounds good to me - if DISPLAY=":1" works, let's ad it to the README.txt and consider this fixed then?

Revision history for this message
kaputtnik (franku) wrote :

Adding it to the README isn't good, because the README.txt is for local testing and setting the environment variable is only needed on the server.

We could deploy this on wl.widelands.org, set the environment variable, run update_help and unset the environment variable again. Doing so we have a updated encyclopedia on the main website.

I would then make another merge proposal for adding a Button to https://wl.widelands.org/encyclopedia/ which is shown only for admins. Clicking this Button would then run update_help,py and the other things for updating the help.

Revision history for this message
GunChleoc (gunchleoc) wrote :

I have never developed any Django stuff before, so I have no idea how long it will take to code such a button - sounds like potentially a lot of work for very little gain. Unless you know how to do it and can do it fast?

And I'm still in favour of documenting this in the readme, so if somebody runs into the same problem in the future when running the updates on the server, they will know what to do instead of banging their head against the wall.

Revision history for this message
kaputtnik (franku) wrote :

Creating such a view isn't as difficult. Problematic would be to redirect the output to such a view. So for now i would suggest another approach: Setting DISPLAY through update_help.py:

# Because wl_map_object_info needs access to a graphical server, which is running at
# DISPLAY :1 we have to set the environment variable accordingly.

if not 'DISPLAY' in os.environ: # No DISPLAY set -> default on the server console
    display_set = True
    os.environ['DISPLAY']=":1"

# Here is the call to wl_map_object_info

if display_set: # Unset DISPLAY
    os.environ.pop('DISPLAY')

This is testet on the alpha site and works fine. A button could be implemented later on.

I think documenting any server related spacial things in the readme would blow it up without a need for normal users which may just want to look into the code and runs the widelands homepage local. Documenting this in the code is imho better.

Revision history for this message
kaputtnik (franku) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'templates/wlhelp/index.html'
2--- templates/wlhelp/index.html 2015-05-23 16:12:31 +0000
3+++ templates/wlhelp/index.html 2016-05-22 08:48:29 +0000
4@@ -10,8 +10,6 @@
5 {% block content %}
6 <h1>Encyclopedia</h1>
7 <div class="blogEntry">
8-<p><span style="font-weight: bold;"> !!! The encyclopedia currently contains some errors. !!!</span><br />Please use the in-game help for correct and up to date information.</p>
9-
10 Encyclopedia &#187; Index
11 <br /><br />
12 <p>This is a list of all tribes in Widelands:</p>
13
14=== modified file 'templates/wlhelp/inlines/display_buildings.html'
15--- templates/wlhelp/inlines/display_buildings.html 2012-05-19 19:55:15 +0000
16+++ templates/wlhelp/inlines/display_buildings.html 2016-05-22 08:48:29 +0000
17@@ -10,7 +10,7 @@
18 <tr class="{% cycle "odd" "even" %}">
19 <td>
20 <a href="{% url wlhelp_building_details b.tribe.name b.name %}" title="{{ b.displayname }}" id="{{ b.name }}">
21- {{ b.displayname|title }}
22+ {{ b.displayname }}
23 <br />
24 <img alt="{{b.displayname}}" src="{{ b.image_url }}" />
25 </a>
26
27=== modified file 'widelandslib/make_flow_diagram.py'
28--- widelandslib/make_flow_diagram.py 2011-07-06 23:31:10 +0000
29+++ widelandslib/make_flow_diagram.py 2016-05-22 08:48:29 +0000
30@@ -3,8 +3,8 @@
31
32 import pydot as d
33
34+from settings import MEDIA_ROOT
35 from widelandslib.tribe import *
36-#from pudb import set_trace; set_trace()
37
38 from os import makedirs, path
39 import subprocess
40@@ -15,12 +15,12 @@
41 ##############################
42 # To Do Make_Flow_Diagram.py #
43 ##############################
44-# i'd like to add things like: forester resores trunk or: gamekeeper restores meat
45+# i'd like to add things like: forester resores log or: gamekeeper restores meat
46 #
47 # also, a building called construction site where alle the building material can point at would be nice
48 #
49 # how to tell the viewer, how many and wich ressources turn to others via buildings,
50-# e.g. 6 trunks to 1 coal with the atlanteans, maybe on the edges?
51+# e.g. 6 logs to 1 coal with the atlanteans, maybe on the edges?
52
53 #############################
54 # Work around bugs in pydot #
55@@ -123,7 +123,7 @@
56
57
58 if isinstance(b, (ProductionSite,)):
59- # for worker,c in b.workers:
60+ # for worker,c in b.workers:
61 # g.add_edge(Edge(worker, name, color="orange"))
62
63 for output in b.outputs:
64@@ -169,7 +169,12 @@
65 def make_graph(tribe_name):
66 global tdir
67 tdir = mkdtemp(prefix="widelands-help")
68- t = Tribe(tribe_name)
69+
70+ json_directory = path.normpath(MEDIA_ROOT + "/map_object_info")
71+ tribeinfo_file = open(path.normpath(json_directory + "/tribe_" + tribe_name + ".json"), "r")
72+ tribeinfo = json.load(tribeinfo_file)
73+
74+ t = Tribe(tribeinfo, json_directory)
75
76 g = CleanedDot(concentrate="false", style="filled", bgcolor="white",
77 overlap="false", splines="true", rankdir="LR")
78@@ -186,7 +191,7 @@
79
80 g.write_pdf(path.join(tdir, "%s.pdf" % tribe_name))
81
82- g.set_size("6")
83+ g.set_size("32")
84 g.write_gif(path.join(tdir, "%s.gif" % tribe_name))
85
86 rtdir, tdir = tdir, ""
87@@ -235,8 +240,8 @@
88 g = CleanedDot(concentrate="false", bgcolor="transparent",
89 overlap="false", splines="true", rankdir="LR")
90
91- buildings = [bld for bld in t.buildings.values() if
92- isinstance(bld, ProductionSite) and
93+ buildings = [bld for bld in t.buildings.values() if
94+ isinstance(bld, ProductionSite) and
95 (w.name in bld.workers or w.name in bld.recruits)]
96
97 for bld in buildings:
98@@ -280,9 +285,9 @@
99 g.write(path.join(tdir, "help/%s/wares/%s/source.dot" % (t.name, ware_name)))
100
101 def process_dotfile(directory):
102- subprocess.Popen(("dot -Tpng -o %s/image.png -Tcmapx -o %s/map.map %s/source.dot" % (directory, directory, directory)).split(" ")).wait()
103+ subprocess.Popen(("dot -Tpng -o %s/menu.png -Tcmapx -o %s/map.map %s/source.dot" % (directory, directory, directory)).split(" ")).wait()
104 #with open(directory,"w") as html:
105- # html.write(r"""<IMG SRC="image.png" border="0px" usemap="#G"/>""" + open(path.join(directory, "map.map")).read())
106+ # html.write(r"""<IMG SRC="menu.png" border="0px" usemap="#G"/>""" + open(path.join(directory, "map.map")).read())
107
108 def make_all_subgraphs(t):
109 global tdir
110@@ -291,6 +296,13 @@
111 t = Tribe(t)
112 print "making all subgraphs for tribe", t.name, "in", tdir
113
114+ print " making wares"
115+
116+ for w in t.wares:
117+ print " " + w
118+ make_ware_graph(t, w)
119+ process_dotfile(path.join(tdir, "help/%s/wares/%s/" % (t.name, w)))
120+
121 print " making workers"
122
123 for w in t.workers:
124@@ -298,13 +310,6 @@
125 make_worker_graph(t, w)
126 process_dotfile(path.join(tdir, "help/%s/workers/%s/" % (t.name, w)))
127
128- print " making wares"
129-
130- for w in t.wares:
131- print " " + w
132- make_ware_graph(t, w)
133- process_dotfile(path.join(tdir, "help/%s/wares/%s/" % (t.name, w)))
134-
135 print " making buildings"
136
137 for b in t.buildings:
138
139=== modified file 'widelandslib/tribe.py'
140--- widelandslib/tribe.py 2012-05-18 23:41:26 +0000
141+++ widelandslib/tribe.py 2016-05-22 08:48:29 +0000
142@@ -1,13 +1,8 @@
143 #!/usr/bin/env python
144 # encoding: utf-8
145
146-from conf import WidelandsConfigParser
147-from ConfigParser import NoSectionError, NoOptionError
148-import conf
149-from itertools import chain
150 import os.path as p
151-import re
152-from string import replace
153+import json
154 try:
155 from settings import WIDELANDS_SVN_DIR
156 basedir = WIDELANDS_SVN_DIR
157@@ -15,18 +10,15 @@
158 basedir = p.join(p.dirname(__file__), p.pardir, p.pardir)
159
160 class BaseDescr(object):
161- def __init__(self, tribe, name, descname, tdir):
162+ def __init__(self, tribe, name, descname, json):
163 self.tribe = tribe
164- self._tdir = tdir
165- self._conf_file = p.join(tdir, name, "conf")
166- self._conf = WidelandsConfigParser(p.join(tdir,name,"conf"))
167-
168+ self._json = json
169 self.name = name
170 self.descname = descname
171
172 @property
173 def image(self):
174- return p.abspath(p.join(self._tdir,self.name,"menu.png"))
175+ return p.abspath(p.join(WIDELANDS_SVN_DIR, "data", self._json['icon']))
176
177 class Ware(BaseDescr):
178 def __str__(self):
179@@ -34,19 +26,11 @@
180
181 class Worker(BaseDescr):
182 @property
183- def outputs(self):
184- rv = set(sorted(
185- i.strip() for i in re.findall(r'\d+=\s*createitem\s*(\w+)',
186- open(self._conf_file).read())
187- ))
188- return rv
189-
190- @property
191 def becomes(self):
192- try:
193- return self._conf.get("global", "becomes")
194- except NoOptionError:
195- return None
196+ if 'becomes' in self._json:
197+ return self._json['becomes']['name']
198+ else:
199+ return None
200
201 def __str__(self):
202 return "Worker(%s)" % self.name
203@@ -54,7 +38,10 @@
204 class Building(BaseDescr):
205 @property
206 def enhanced_building(self):
207- return self._conf.getboolean("global", "enhanced_building", False)
208+ if 'enhanced' in self._json:
209+ return True
210+ else:
211+ return False
212
213 @property
214 def base_building(self):
215@@ -69,110 +56,113 @@
216
217 @property
218 def enhancement(self):
219- rv = self._conf.getstring("global", "enhancement", "none")
220- return rv if rv != "none" else None
221-
222- @property
223- def image(self):
224- glob_pat = self._conf.getstring("idle", "pics")
225- return p.abspath(p.join(self._tdir,self.name,replace(glob_pat,'?','0')))
226+ if 'enhancement' in self._json:
227+ return self._json['enhancement']
228+ else:
229+ return None
230
231 @property
232 def buildcost(self):
233- try:
234- return dict(self._conf.items("buildcost"))
235- except NoSectionError:
236- return {}
237+ result = dict()
238+ if 'buildcost' in self._json:
239+ for buildcost in self._json['buildcost']:
240+ result[buildcost['name']] = buildcost['amount']
241+ return result
242
243 @property
244 def size(self):
245- return self._conf.getstring("global", "size")
246+ return self._json['size']
247
248 class ProductionSite(Building):
249 btype = "productionsite"
250 @property
251 def outputs(self):
252- self_produced = set(sorted(
253- i.strip() for i in re.findall(r'produce\s*=\s*(\w+)',
254- open(self._conf_file).read())
255- ))
256- if not len(self_produced):
257- rv = reduce(lambda a,b: a | b, [ self.tribe.workers[w].outputs
258- for w in self.workers ], set())
259- return rv
260- return self_produced
261+ result = set()
262+ if 'produced_wares' in self._json:
263+ for warename in self._json['produced_wares']:
264+ result.add(warename)
265+ return result
266
267 @property
268 def inputs(self):
269- try:
270- return dict( (k, v) for k,v in self._conf.items("inputs") )
271- except conf.NoSectionError:
272- return dict()
273+ result = dict()
274+ if 'stored_wares' in self._json:
275+ for ware in self._json['stored_wares']:
276+ result[ware['name']] = ware['amount']
277+ return result
278
279 @property
280 def workers(self):
281- return dict( (k, v) for k,v in self._conf.items("working positions") )
282+ result = dict()
283+ if 'workers' in self._json:
284+ for worker in self._json['workers']:
285+ result[worker['name']] = worker['amount']
286+ return result
287
288 @property
289 def recruits(self):
290- recs = set([])
291- for prog,_ in self._conf.items("programs"):
292- recs |= set([name for type, name in self._conf.items(prog) if type == "recruit"])
293- return recs
294+ result = set()
295+ if 'produced_workers' in self._json:
296+ for workername in self._json['produced_workers']:
297+ result.add(workername)
298+ return result
299
300 class Warehouse(Building):
301 btype = "warehouse"
302 pass
303
304 class TrainingSite(ProductionSite):
305- btype = "trainings site"
306+ btype = "trainingsite"
307 pass
308
309 class MilitarySite(Building):
310- btype = "military site"
311+ btype = "militarysite"
312 @property
313 def conquers(self):
314- return self._conf.get("global", "conquers")
315+ return self._json['conquers']
316
317 @property
318 def max_soldiers(self):
319- return self._conf.get("global", "max_soldiers")
320+ return self._json['max_soldiers']
321
322 @property
323 def heal_per_second(self):
324- return self._conf.getint("global", "heal_per_second")
325+ return self._json['heal_per_second']
326
327
328 class Tribe(object):
329- def __init__(self, name, bdir = basedir):
330- self.name = name
331-
332- self._tdir = p.join(bdir, "tribes", name)
333-
334- self._conf = WidelandsConfigParser(p.join(self._tdir, "conf"))
335-
336- self.wares = dict( (k,Ware(self, k, v, self._tdir)) for k,v in
337- self._conf.items("ware types"))
338- self.workers = dict(chain(
339- ((k,Worker(self, k, v, self._tdir)) for k,v in
340- self._conf.items("worker types")),
341- ((k,Worker(self, k, v, self._tdir)) for k,v in
342- self._conf.items("carrier types")),
343- ))
344-
345-
346- self.buildings = dict(chain(
347- ((k,ProductionSite(self, k, v, self._tdir)) for k,v in \
348- self._conf.items("productionsite types")),
349- ((k,MilitarySite(self, k, v, self._tdir)) for k,v in \
350- self._conf.items("militarysite types")),
351- ((k,Warehouse(self, k, v, self._tdir)) for k,v in \
352- self._conf.items("warehouse types")),
353- ((k,TrainingSite(self, k, v, self._tdir)) for k,v in \
354- self._conf.items("trainingsite types")),
355- ))
356+ def __init__(self, tribeinfo, json_directory):
357+ self.name = tribeinfo['name']
358+
359+ wares_file = open(p.normpath(json_directory + "/" + self.name + "_wares.json"), "r")
360+ waresinfo = json.load(wares_file)
361+ self.wares = dict()
362+ for ware in waresinfo['wares']:
363+ descname = ware['descname'].encode('ascii', 'xmlcharrefreplace')
364+ self.wares[ware['name']] = Ware(self, ware['name'], descname, ware)
365+
366+ workers_file = open(p.normpath(json_directory + "/" + self.name + "_workers.json"), "r")
367+ workersinfo = json.load(workers_file)
368+ self.workers = dict()
369+ for worker in workersinfo['workers']:
370+ descname = worker['descname'].encode('ascii', 'xmlcharrefreplace')
371+ self.workers[worker['name']] = Worker(self, worker['name'], descname, worker)
372+
373+ buildings_file = open(p.normpath(json_directory + "/" + self.name + "_buildings.json"), "r")
374+ buildingsinfo = json.load(buildings_file)
375+ self.buildings = dict()
376+ for building in buildingsinfo['buildings']:
377+ descname = building['descname'].encode('ascii', 'xmlcharrefreplace')
378+ if building['type'] == "productionsite":
379+ self.buildings[building['name']] = ProductionSite(self, building['name'], descname, building)
380+ elif building['type'] == "warehouse":
381+ self.buildings[building['name']] = Warehouse(self, building['name'], descname, building)
382+ elif building['type'] == "trainingsite":
383+ self.buildings[building['name']] = TrainingSite(self, building['name'], descname, building)
384+ elif building['type'] == "militarysite":
385+ self.buildings[building['name']] = MilitarySite(self, building['name'], descname, building)
386+ else:
387+ self.buildings[building['name']] = Building(self, building['name'], descname, building)
388
389 def __str__(self):
390 return "Tribe(%s)" % self.name
391-
392-
393
394=== modified file 'wlhelp/management/commands/update_help.py'
395--- wlhelp/management/commands/update_help.py 2012-05-19 19:55:15 +0000
396+++ wlhelp/management/commands/update_help.py 2016-05-22 08:48:29 +0000
397@@ -16,230 +16,313 @@
398
399 from django.core.files import File
400 from django.core.management.base import BaseCommand, CommandError
401-from optparse import make_option
402
403-from ConfigParser import ConfigParser, MissingSectionHeaderError
404-from glob import glob
405 import os
406-from os import path
407+from os import makedirs, path
408 import shutil
409-from cStringIO import StringIO
410 import re
411-from itertools import chain
412+import json
413+import subprocess
414
415 from settings import MEDIA_ROOT, WIDELANDS_SVN_DIR, MEDIA_URL
416
417 from widelandslib.tribe import *
418 from widelandslib.make_flow_diagram import make_all_subgraphs
419
420-def normalize_name( s ):
421- """
422- Strips _ from the name endings
423- """
424- return s.strip('_')
425
426 class TribeParser(object):
427- map_mouseover_pattern = re.compile(r'(?P<beginning>.*href="../../(?P<type>[^/]+)s/(?P<name>[^/]+)/".*")&lt;TABLE&gt;(?P<rest>.*)')
428+ map_mouseover_pattern = re.compile(
429+ r'(?P<beginning>.*href="../../(?P<type>[^/]+)s/(?P<name>[^/]+)/".*")&lt;TABLE&gt;(?P<rest>.*)')
430+
431 def __init__(self, name):
432- """
433- Parses the definitions for one tribe and generates the models
434+ """Parses the definitions for one tribe and generates the models.
435
436 name - name of the tribe
437- conf - path to the tribe/conf file
438+
439 """
440- self._tribe = Tribe(name)
441+ self._delete_old_media_dir(
442+ name) # You can deactivate this line if you don't need to clean house.
443+
444+ base_directory = os.path.normpath(WIDELANDS_SVN_DIR + '/data')
445+ json_directory = os.path.normpath(MEDIA_ROOT + '/map_object_info')
446+
447+ tribeinfo_file = open(os.path.normpath(
448+ json_directory + '/tribe_' + name + '.json'), 'r')
449+ tribeinfo = json.load(tribeinfo_file)
450+
451+ self._tribe = Tribe(tribeinfo, json_directory)
452 # Generate the Tribe
453 self._to = TribeModel.objects.get_or_create(name=name.lower())[0]
454- self._to.displayname = normalize_name(self._tribe._conf.getstring("tribe", "name"))
455- self._to.descr = normalize_name(self._tribe._conf.getstring("tribe", "descr"))
456+ self._to.displayname = tribeinfo['descname']
457+ self._to.descr = tribeinfo['tooltip']
458 # copy icon
459- dn = "%s/wlhelp/img/%s/" % (MEDIA_ROOT,self._to.name)
460+ dn = os.path.normpath('%s/wlhelp/img/%s/' %
461+ (MEDIA_ROOT, tribeinfo['name']))
462 try:
463 os.makedirs(dn)
464 except OSError, o:
465 if o.errno != 17:
466 raise
467- new_name = path.join(dn, "icon.png")
468- file = path.join(self._tribe._tdir,self._tribe._conf.getstring("tribe", "icon"))
469- shutil.copy(file, new_name )
470- self._to.icon_url = path.normpath("%s/%s" % (MEDIA_URL, new_name[len(MEDIA_ROOT):]))
471+ new_name = path.join(dn, 'icon.png')
472+ file = os.path.normpath(base_directory + '/' + tribeinfo['icon'])
473+ shutil.copy(file, new_name)
474+ self._to.icon_url = path.normpath(
475+ '%s/%s' % (MEDIA_URL, new_name[len(MEDIA_ROOT):]))
476 self._to.save()
477
478- def parse( self ):
479- """Put all data into the database"""
480- #self._delete_old_media_dir() why delete it? We can simply overwrite data
481- self._parse_workers()
482- self._parse_wares()
483- self._parse_buildings()
484-
485- def graph( self ):
486- """Make all graphs"""
487+ def parse(self, tribename, base_directory, json_directory):
488+ """Put all data into the database."""
489+ self._delete_old_data(
490+ tribename) # You can deactivate this line if you don't need to clean house.
491+
492+ wares_file = open(os.path.normpath(
493+ json_directory + '/' + tribename + '_wares.json'), 'r')
494+ self._parse_wares(base_directory, json.load(wares_file))
495+
496+ workers_file = open(os.path.normpath(
497+ json_directory + '/' + tribename + '_workers.json'), 'r')
498+ self._parse_workers(base_directory, json.load(workers_file))
499+
500+ buildings_file = open(os.path.normpath(
501+ json_directory + '/' + tribename + '_buildings.json'), 'r')
502+ self._parse_buildings(base_directory, json.load(buildings_file))
503+
504+ def graph(self):
505+ """Make all graphs."""
506 tdir = make_all_subgraphs(self._tribe)
507- for obj, cls in [(WorkerModel, "workers"),
508- (BuildingModel, "buildings"),
509- (WareModel, "wares")]:
510+ for obj, cls in [(WorkerModel, 'workers'),
511+ (BuildingModel, 'buildings'),
512+ (WareModel, 'wares')]:
513 for inst in obj.objects.all().filter(tribe=self._to):
514 try:
515- fpath = path.join(tdir,"help/%s/%s/%s/" % (self._tribe.name, cls, inst.name))
516- url = self._copy_picture(path.join(fpath, "image.png"), inst.name, "graph.png")
517+ fpath = path.join(tdir, 'help/%s/%s/%s/' %
518+ (self._tribe.name, cls, inst.name))
519+ url = self._copy_picture(
520+ path.join(fpath, 'menu.png'), inst.name, 'graph.png')
521 inst.graph_url = url
522- inst.imagemap = open(path.join(fpath, "map.map")).read()
523- inst.imagemap = self.map_mouseover_pattern.sub(r"\1Show the \2 \3\4", inst.imagemap)
524+ inst.imagemap = open(path.join(fpath, 'map.map')).read()
525+ inst.imagemap = self.map_mouseover_pattern.sub(
526+ r"\1Show the \2 \3\4", inst.imagemap)
527 inst.save()
528 except Exception, e:
529- print "Exception while handling", cls, "of", self._tribe.name, ":", inst.name
530+ print 'Exception while handling', cls, 'of', self._tribe.name, ':', inst.name
531 print type(e), e, repr(e)
532-
533+
534 shutil.rmtree(tdir)
535
536- def _delete_old_media_dir(self):
537- sdir = os.path.join(MEDIA_ROOT, "wlhelp/img", self._to.name)
538+ def _delete_old_media_dir(self, tribename):
539+ """Clean house, e.g. when we have renamed a map object."""
540+
541+ print('Deleting old media files...')
542+ sdir = os.path.normpath(os.path.join(
543+ MEDIA_ROOT, 'wlhelp/img', tribename))
544 if os.path.exists(sdir):
545 shutil.rmtree(sdir)
546
547- def _copy_picture( self, file, name, fname ):
548- """
549- Copy the given image into the media directory
550+ def _delete_old_data(self, tribename):
551+ """Clean house, e.g. when we have renamed a map object."""
552+
553+ t = TribeModel.objects.get(name=tribename)
554+ print('Deleting old wares...')
555+ for ware in WareModel.objects.filter(tribe=t):
556+ ware.delete()
557+ print('Deleting old workers...')
558+ for worker in WorkerModel.objects.filter(tribe=t):
559+ worker.delete()
560+ print('Deleting old buildings...')
561+ for building in BuildingModel.objects.filter(tribe=t):
562+ building.delete()
563+
564+ def _copy_picture(self, file, name, fname):
565+ """Copy the given image into the media directory.
566
567 file - original path of image
568 name - name of the item (coal, iron...)
569 fname - file name of the picture
570+
571 """
572- dn = "%s/wlhelp/img/%s/%s/" % (MEDIA_ROOT,self._to.name,name)
573+ dn = os.path.normpath('%s/wlhelp/img/%s/%s/' %
574+ (MEDIA_ROOT, self._to.name, name))
575 try:
576 os.makedirs(dn)
577 except OSError, o:
578 if o.errno != 17:
579 raise
580 new_name = path.join(dn, fname)
581- shutil.copy(file, new_name )
582-
583- return "%s/%s" % (MEDIA_URL, new_name[len(MEDIA_ROOT):])
584-
585- def _parse_workers( self ):
586- """Put the workers into the database"""
587- print " parsing workers"
588- for worker in self._tribe.workers.values():
589- print " " + worker.name
590- nn = self._copy_picture(worker.image, worker.name, "menu.png")
591-
592- workero = WorkerModel.objects.get_or_create( tribe = self._to, name = worker.name )[0]
593- workero.displayname = normalize_name(worker.descname)
594+ shutil.copy(file, new_name)
595+
596+ return '%s/%s' % (MEDIA_URL, new_name[len(MEDIA_ROOT):])
597+
598+ def _parse_workers(self, base_directory, workersinfo):
599+ """Put the workers into the database."""
600+ print ' parsing workers'
601+
602+ for worker in workersinfo['workers']:
603+ print ' ' + worker['name']
604+ nn = self._copy_picture(os.path.normpath(
605+ base_directory + '/' + worker['icon']), worker['name'], 'menu.png')
606+
607+ workero = WorkerModel.objects.get_or_create(
608+ tribe=self._to, name=worker['name'])[0]
609+ workero.displayname = worker['descname']
610 workero.image_url = nn
611
612- # See if there is help available
613- if worker._conf.has_option("global","help"):
614- helpstr = normalize_name(worker._conf.get("global","help"))
615- workero.help = helpstr
616+ # Help
617+ workero.help = worker['helptext']
618
619 # Check for experience
620- if worker._conf.has_option("global","experience"):
621- experience = normalize_name(worker._conf.get("global","experience"))
622- workero.exp = experience
623+ if 'experience' in worker:
624+ workero.exp = worker['experience']
625
626 # See what the worker becomes
627- try:
628- enname = worker.becomes
629- workero.becomes = WorkerModel.objects.get_or_create(
630- name=enname, tribe = self._to)[0]
631- except:
632- pass
633+ if 'becomes' in worker:
634+ try:
635+ enname = worker.becomes
636+ workero.becomes = WorkerModel.objects.get_or_create(
637+ name=worker['becomes']['name'], tribe=self._to)[0]
638+ except:
639+ pass
640
641 workero.save()
642
643- def _parse_wares( self ):
644- print " parsing wares"
645- for ware in self._tribe.wares.values():
646- print " " + ware.name
647- nn = self._copy_picture(ware.image, ware.name, "menu.png")
648-
649- w = WareModel.objects.get_or_create( tribe = self._to, name = ware.name )[0]
650- w.displayname = normalize_name(ware.descname)
651+ def _parse_wares(self, base_directory, waresinfo):
652+ print ' parsing wares'
653+
654+ for ware in waresinfo['wares']:
655+ print ' ' + ware['name']
656+ nn = self._copy_picture(os.path.normpath(
657+ base_directory + '/' + ware['icon']), ware['name'], 'menu.png')
658+
659+ w = WareModel.objects.get_or_create(
660+ tribe=self._to, name=ware['name'])[0]
661+ w.displayname = ware['descname']
662 w.image_url = nn
663
664-
665- # See if there is help available
666- if ware._conf.has_option("global","help"):
667- helpstr = normalize_name(ware._conf.get("global","help"))
668- w.help = helpstr
669+ # Help
670+ w.help = ware['helptext']
671
672 w.save()
673
674- def _parse_buildings( self ):
675- def objects_with_counts(objtype, set_):
676- counts = ' '.join(set_.values())
677- objects = [objtype.objects.get_or_create(name = w, tribe = self._to)[0] for w in set_.keys()]
678+ def _parse_buildings(self, base_directory, buildingsinfo):
679+ def objects_with_counts(objtype, json_):
680+ element_set = {}
681+ for element in json_:
682+ element_set[element['name']] = str(element['amount'])
683+ counts = ' '.join(element_set.values())
684+ objects = [objtype.objects.get_or_create(name=w, tribe=self._to)[
685+ 0] for w in element_set.keys()]
686 return counts, objects
687
688- enhancement_hier = []
689- print " parsing buildings"
690- for building in self._tribe.buildings.values():
691- print " " + building.name
692- b = BuildingModel.objects.get_or_create( tribe = self._to, name = building.name )[0]
693- b.displayname = normalize_name(building.descname)
694- b.type = building.btype
695+ enhancement_hierarchy = []
696+ print ' parsing buildings'
697+
698+ for building in buildingsinfo['buildings']:
699+ print ' ' + building['name']
700+ b = BuildingModel.objects.get_or_create(
701+ tribe=self._to, name=building['name'])[0]
702+ b.displayname = building['descname']
703+ b.type = building['type']
704
705 # Get the building size
706- size = building.size
707- res, = [ k for k,v in BuildingModel.SIZES if v == size ]
708-
709+ size = building['size']
710+ res, = [k for k, v in BuildingModel.SIZES if v == size]
711 b.size = res
712
713- nn = self._copy_picture(building.image, building.name, "menu.png" )
714+ nn = self._copy_picture(os.path.normpath(
715+ base_directory + '/' + building['icon']), building['name'], 'menu.png')
716 b.image_url = nn
717
718- # Try to figure out buildcost
719- b.build_costs, b.build_wares = objects_with_counts(WareModel, building.buildcost)
720+ # Buildcost if we have any
721+ if 'buildcost' in building:
722+ b.build_costs, b.build_wares = objects_with_counts(
723+ WareModel, building['buildcost'])
724
725 # Try to figure out who works there
726- if isinstance(building, ProductionSite):
727- b.workers_count, b.workers_types = objects_with_counts(WorkerModel, building.workers)
728-
729- # Try to figure out if this is an enhanced building
730- if building.enhancement:
731- enhancement_hier.append((b, building.enhancement))
732-
733- if building._conf.has_option("global","help"):
734- b.help = normalize_name(building._conf.get("global","help"))
735- else:
736- try:
737- b.help = [worker.help for worker in b.workers_types.all()][0]
738- except IndexError:
739- print "could not find a help string for %s (%s) anywhere!" % (b.name, self._tribe.name)
740-
741- # See if there is any inputs field around
742- if isinstance(building, ProductionSite):
743- b.store_count, b.store_wares = objects_with_counts(WareModel, building.inputs)
744-
745- b.output_wares = [WareModel.objects.get_or_create(name = w, tribe = self._to)[0] for w in building.outputs]
746-
747- try:
748- b.output_workers = [Worker.objects.get_or_create(name = w, tribe = self._to)[0] for w in building.recruits]
749- except AttributeError:
750- pass
751+ if 'workers' in building:
752+ b.workers_count, b.workers_types = objects_with_counts(
753+ WorkerModel, building['workers'])
754+
755+ # Try to figure out if this building can be enhanced
756+ if 'enhancement' in building:
757+ enhancement_hierarchy.append((b, building['enhancement']))
758+
759+ b.help = building['helptext']
760+
761+ # Input wares
762+ if 'stored_wares' in building:
763+ b.store_count, b.store_wares = objects_with_counts(
764+ WareModel, building['stored_wares'])
765+
766+ # Output wares
767+ if 'produced_wares' in building:
768+ b.output_wares = [WareModel.objects.get_or_create(name=w, tribe=self._to)[
769+ 0] for w in building['produced_wares']]
770+
771+ # Output workers
772+ if 'produced_workers' in building:
773+ b.output_workers = [WorkerModel.objects.get_or_create(
774+ name=w, tribe=self._to)[0] for w in building['produced_workers']]
775
776 b.save()
777
778- for b, tgt in enhancement_hier:
779+ for b, tgt in enhancement_hierarchy:
780 try:
781- b.enhancement = BuildingModel.objects.get(name = tgt, tribe = self._to)
782+ b.enhancement = BuildingModel.objects.get(
783+ name=tgt, tribe=self._to)
784 except Exception, e:
785 raise
786 b.save()
787
788+
789 class Command(BaseCommand):
790 help =\
791- '''Reparses the conf files in a current checkout. '''
792-
793- def handle(self, directory = WIDELANDS_SVN_DIR, **kwargs):
794-
795- tribes = [d for d in glob("%s/tribes/*" % directory)
796- if os.path.isdir(d)]
797-
798- for t in tribes:
799- tribename = os.path.basename(t)
800- print "updating help for tribe ", tribename
801- p = TribeParser(tribename)
802-
803- p.parse()
804- p.graph()
805+ '''Regenerates and parses the json files in a current checkout. '''
806+
807+ def handle(self, directory=os.path.normpath(WIDELANDS_SVN_DIR + '/data'), **kwargs):
808+
809+ json_directory = os.path.normpath(MEDIA_ROOT + '/map_object_info')
810+
811+ if not os.path.exists(json_directory):
812+ os.makedirs(json_directory)
813+
814+ print('JSON files will be written to: ' + json_directory)
815+
816+ # First, we make sure that JSON files have been generated.
817+ current_dir = os.path.dirname(os.path.realpath(__file__))
818+ is_json_valid = False
819+ os.chdir(WIDELANDS_SVN_DIR)
820+ try:
821+ subprocess.check_call(
822+ [os.path.normpath('wl_map_object_info'), json_directory])
823+ except:
824+ print(
825+ "Error: Unable to execute 'wl_map_object_info' for generating the JSON files.")
826+
827+ # Now we validate that they are indeed JSON files (syntax check only)
828+ validator_script = os.path.normpath(
829+ WIDELANDS_SVN_DIR + '/utils/validate_json.py')
830+ if not os.path.isfile(validator_script):
831+ print("Wrong path for 'utils/validate_json.py': " +
832+ validator_script + ' does not exist!')
833+ try:
834+ subprocess.check_call(
835+ [validator_script, json_directory])
836+ is_json_valid = True
837+ except:
838+ print('Error: JSON files are not valid.')
839+
840+ os.chdir(current_dir)
841+
842+ # We regenerate the encyclopedia only if the JSON files passed the
843+ # syntax check
844+ if is_json_valid:
845+ source_file = open(os.path.normpath(
846+ json_directory + '/tribes.json'), 'r')
847+ tribesinfo = json.load(source_file)
848+
849+ for t in tribesinfo['tribes']:
850+ tribename = t['name']
851+ print 'updating help for tribe ', tribename
852+ p = TribeParser(tribename)
853+ p.parse(tribename, directory, json_directory)
854+ p.graph()
855
856=== modified file 'wlhelp/management/commands/update_help_pdf.py'
857--- wlhelp/management/commands/update_help_pdf.py 2012-05-19 19:55:15 +0000
858+++ wlhelp/management/commands/update_help_pdf.py 2016-05-22 08:48:29 +0000
859@@ -10,25 +10,29 @@
860 from widelandslib.make_flow_diagram import make_graph
861 from wlhelp.models import Tribe
862 from glob import glob
863+import json
864+
865
866 class Command(BaseCommand):
867 help =\
868- """Update the overview pdfs of all tribes in a current checkout"""
869-
870- def handle(self, directory = WIDELANDS_SVN_DIR, **kwargs):
871- tribes = [d for d in glob("%s/tribes/*" % directory)
872- if os.path.isdir(d)]
873-
874- print "updating pdf files for all tribes"
875-
876- for t in tribes:
877- tribename = os.path.basename(t)
878- print " updating pdf file for tribe ", tribename
879+ """Update the overview pdfs of all tribes in a current checkout"""
880+
881+ def handle(self, json_directory=os.path.normpath(MEDIA_ROOT + '/map_object_info'), **kwargs):
882+ source_file = open(os.path.normpath(
883+ json_directory + '/tribes.json'), 'r')
884+ tribesinfo = json.load(source_file)
885+
886+ print 'updating pdf files for all tribes'
887+
888+ for t in tribesinfo['tribes']:
889+ tribename = t['name']
890+ print ' updating pdf file for tribe ', tribename
891 gdir = make_graph(tribename)
892- pdffile = path.join(gdir, tribename + ".pdf")
893- giffile = path.join(gdir, tribename + ".gif")
894+ pdffile = path.join(gdir, tribename + '.pdf')
895+ giffile = path.join(gdir, tribename + '.gif')
896
897- targetdir = path.join(MEDIA_ROOT, "wlhelp", "network_graphs", tribename)
898+ targetdir = path.normpath(path.join(MEDIA_ROOT, 'wlhelp',
899+ 'network_graphs', tribename))
900
901 try:
902 os.makedirs(targetdir)
903@@ -40,10 +44,12 @@
904
905 tribe = Tribe.objects.get(name=tribename)
906 if tribe:
907- tribe.network_pdf_url = path.normpath("%s/%s/%s" % (MEDIA_URL, targetdir[len(MEDIA_ROOT):], tribename + ".pdf"))
908- tribe.network_gif_url = path.normpath("%s/%s/%s" % (MEDIA_URL, targetdir[len(MEDIA_ROOT):], tribename + ".gif"))
909+ tribe.network_pdf_url = path.normpath(
910+ '%s/%s/%s' % (MEDIA_URL, targetdir[len(MEDIA_ROOT):], tribename + '.pdf'))
911+ tribe.network_gif_url = path.normpath(
912+ '%s/%s/%s' % (MEDIA_URL, targetdir[len(MEDIA_ROOT):], tribename + '.gif'))
913 tribe.save()
914 else:
915- print "Could not set tribe urls"
916+ print 'Could not set tribe urls'
917
918 shutil.rmtree(gdir)
919
920=== modified file 'wlhelp/models.py'
921--- wlhelp/models.py 2012-05-19 19:55:15 +0000
922+++ wlhelp/models.py 2016-05-22 08:48:29 +0000
923@@ -4,35 +4,40 @@
924 if settings.USE_SPHINX:
925 from djangosphinx.models import SphinxSearch
926
927+
928 class Tribe(models.Model):
929 name = models.CharField(max_length=100)
930 displayname = models.CharField(max_length=100)
931 descr = models.TextField()
932- icon_url = models.CharField( max_length=256 )
933- network_pdf_url = models.CharField( max_length=256 )
934- network_gif_url = models.CharField( max_length=256 )
935+ icon_url = models.CharField(max_length=256)
936+ network_pdf_url = models.CharField(max_length=256)
937+ network_gif_url = models.CharField(max_length=256)
938
939
940 class Worker(models.Model):
941 if settings.USE_SPHINX:
942- search = SphinxSearch(
943- weights = {
944+ search = SphinxSearch(
945+ weights={
946 'displayname': 100,
947 'help': 60,
948 'name': 20,
949- }
950+ }
951 )
952
953 name = models.CharField(max_length=100)
954 displayname = models.CharField(max_length=100)
955 tribe = models.ForeignKey(Tribe)
956- image_url = models.CharField( max_length=256 ) # URL to include this, i wasn't able to feed django local images
957- graph_url = models.CharField( max_length=256 ) # URL to the help graph
958- imagemap = models.TextField() # the image map for the help graph
959+ # URL to include this, i wasn't able to feed django local images
960+ image_url = models.CharField(max_length=256)
961+ graph_url = models.CharField(max_length=256) # URL to the help graph
962+ imagemap = models.TextField() # the image map for the help graph
963
964- help = models.TextField(max_length=256) # This limit shall probably cover the longest help (found 209, nothing more)
965- exp = models.TextField(max_length=8) # Just in case
966- becomes = models.OneToOneField('self', related_name="trained_by_experience", blank=True, null=True)
967+ # This limit shall probably cover the longest help (found 209, nothing
968+ # more)
969+ help = models.TextField(max_length=256)
970+ exp = models.TextField(max_length=8) # Just in case
971+ becomes = models.OneToOneField(
972+ 'self', related_name='trained_by_experience', blank=True, null=True)
973
974 def __unicode__(self):
975 return u'%s' % self.name
976@@ -42,43 +47,53 @@
977 name = models.CharField(max_length=100)
978 displayname = models.CharField(max_length=100)
979 tribe = models.ForeignKey(Tribe)
980- image_url = models.CharField( max_length=256 ) # URL to include this, i wasn't able to feed django local images
981- graph_url = models.CharField( max_length=256 ) # URL to the help graph
982- imagemap = models.TextField() # the image map for the help graph
983+ # URL to include this, i wasn't able to feed django local images
984+ image_url = models.CharField(max_length=256)
985+ graph_url = models.CharField(max_length=256) # URL to the help graph
986+ imagemap = models.TextField() # the image map for the help graph
987
988- help = models.TextField(max_length=256) # This limit shall probably cover the longest help (found 209, nothing more)
989+ # This limit shall probably cover the longest help (found 209, nothing
990+ # more)
991+ help = models.TextField(max_length=256)
992
993 if settings.USE_SPHINX:
994- search = SphinxSearch(
995- weights = {
996+ search = SphinxSearch(
997+ weights={
998 'displayname': 100,
999 'help': 60,
1000 'name': 20,
1001- }
1002+ }
1003 )
1004
1005-
1006 def __unicode__(self):
1007 return u'%s' % self.name
1008
1009
1010 class BuildingManager(models.Manager):
1011+
1012 def small(self):
1013- return self.all().filter(size="S")
1014+ return self.all().filter(size='S')
1015+
1016 def medium(self):
1017- return self.all().filter(size="M")
1018+ return self.all().filter(size='M')
1019+
1020 def big(self):
1021- return self.all().filter(size="B")
1022+ return self.all().filter(size='B')
1023+
1024 def mine(self):
1025- return self.all().filter(size="I")
1026+ return self.all().filter(size='I')
1027+
1028 def port(self):
1029- return self.all().filter(size="P")
1030+ return self.all().filter(size='P')
1031
1032+ def headquarters(self):
1033+ return self.all().filter(size='H')
1034
1035 # return self.build_wares.count()
1036
1037 pass
1038
1039+
1040 class Building(models.Model):
1041 SIZES = (
1042 ('S', 'small'),
1043@@ -86,61 +101,71 @@
1044 ('B', 'big'),
1045 ('I', 'mine'),
1046 ('P', 'port'),
1047+ ('H', 'headquarters'),
1048 )
1049 TYPES = (
1050 ('P', 'productionsite'),
1051 ('W', 'warehouse'),
1052- ('M', 'military site'),
1053- ('T', 'trainings site'),
1054+ ('M', 'militarysite'),
1055+ ('T', 'trainingsite')
1056 )
1057-
1058+
1059 objects = BuildingManager()
1060-
1061+
1062 if settings.USE_SPHINX:
1063- search = SphinxSearch(
1064- weights = {
1065+ search = SphinxSearch(
1066+ weights={
1067 'displayname': 100,
1068 'help': 60,
1069 'name': 20,
1070- }
1071+ }
1072 )
1073
1074-
1075 name = models.CharField(max_length=100)
1076 displayname = models.CharField(max_length=100)
1077 tribe = models.ForeignKey(Tribe)
1078- image_url = models.CharField( max_length=256 ) # URL to include this, i wasn't able to feed django local images
1079- graph_url = models.CharField( max_length=256 ) # URL to the help graph
1080- imagemap = models.TextField() # the image map for the help graph
1081+ # URL to include this, i wasn't able to feed django local images
1082+ image_url = models.CharField(max_length=256)
1083+ graph_url = models.CharField(max_length=256) # URL to the help graph
1084+ imagemap = models.TextField() # the image map for the help graph
1085
1086- size = models.CharField(max_length=1,choices=SIZES)
1087- type = models.CharField( max_length=1, choices=TYPES) # productionsite...
1088+ size = models.CharField(max_length=1, choices=SIZES)
1089+ type = models.CharField(max_length=1, choices=TYPES) # productionsite...
1090
1091 help = models.TextField(blank=True)
1092
1093 # Enhances to
1094- enhancement = models.OneToOneField('self', related_name='enhanced_from', blank=True, null=True)
1095+ enhancement = models.OneToOneField(
1096+ 'self', related_name='enhanced_from', blank=True, null=True)
1097
1098 # Build cost
1099- build_wares = models.ManyToManyField(Ware, related_name="build_ware_for_buildings", blank=True)
1100- build_costs = models.CharField(max_length=100, blank=True) # ' '.joined() integer strings
1101+ build_wares = models.ManyToManyField(
1102+ Ware, related_name='build_ware_for_buildings', blank=True)
1103+ # ' '.joined() integer strings
1104+ build_costs = models.CharField(max_length=100, blank=True)
1105
1106 # Workers
1107- workers_types = models.ManyToManyField(Worker, related_name="workers_for_buildings", blank=True)
1108- workers_count = models.CharField(max_length=100, blank=True) # ' '.joined() integer strings
1109+ workers_types = models.ManyToManyField(
1110+ Worker, related_name='workers_for_buildings', blank=True)
1111+ # ' '.joined() integer strings
1112+ workers_count = models.CharField(max_length=100, blank=True)
1113
1114 # Store
1115- store_wares = models.ManyToManyField(Ware, related_name="stored_ware_for_buildings", blank=True)
1116- store_count = models.CharField(max_length=100, blank=True) # ' '.joined() integer strings
1117+ store_wares = models.ManyToManyField(
1118+ Ware, related_name='stored_ware_for_buildings', blank=True)
1119+ # ' '.joined() integer strings
1120+ store_count = models.CharField(max_length=100, blank=True)
1121
1122 # Output
1123- output_wares = models.ManyToManyField(Ware, related_name="produced_by_buildings", blank=True)
1124- output_workers = models.ManyToManyField(Worker, related_name="trained_by_buildings", blank=True)
1125+ output_wares = models.ManyToManyField(
1126+ Ware, related_name='produced_by_buildings', blank=True)
1127+ output_workers = models.ManyToManyField(
1128+ Worker, related_name='trained_by_buildings', blank=True)
1129
1130 def save(self, *args, **kwargs):
1131
1132- tdict = dict((b,a) for a,b in self.TYPES)
1133- sdict = dict((b,a) for a,b in self.SIZES)
1134+ tdict = dict((b, a) for a, b in self.TYPES)
1135+ sdict = dict((b, a) for a, b in self.SIZES)
1136
1137 self.type = tdict.get(self.type, self.type)
1138 self.size = sdict.get(self.size, self.size)
1139@@ -149,37 +174,42 @@
1140
1141 def has_build_cost(self):
1142 return (self.build_wares.all().count() != 0)
1143+
1144 def get_build_cost(self):
1145- count = map(int,self.build_costs.split( ))
1146- for c,w in zip(count,self.build_wares.all()):
1147- yield [w]*c
1148+ count = map(int, self.build_costs.split())
1149+ for c, w in zip(count, self.build_wares.all()):
1150+ yield [w] * c
1151
1152 def has_workers(self):
1153 return (self.workers_types.all().count() != 0)
1154+
1155 def get_workers(self):
1156- count = map(int,self.workers_count.split( ))
1157- for c,wor in zip(count,self.workers_types.all()):
1158- yield [wor]*c
1159+ count = map(int, self.workers_count.split())
1160+ for c, wor in zip(count, self.workers_types.all()):
1161+ yield [wor] * c
1162
1163 def produces(self):
1164 return (self.output_wares.all().count() != 0)
1165+
1166 def get_ware_outputs(self):
1167 return self.output_wares.all()
1168+
1169 def trains(self):
1170 return (self.output_workers.all().count() != 0)
1171+
1172 def get_worker_outputs(self):
1173 return self.output_workers.all()
1174+
1175 def has_outputs(self):
1176 return (self.output_workers.all().count() != 0 or self.output_wares.all().count() != 0)
1177
1178 def has_stored_wares(self):
1179 return (self.store_wares.all().count() != 0)
1180+
1181 def get_stored_wares(self):
1182- count = map(int,self.store_count.split( ))
1183- for c,w in zip(count,self.store_wares.all()):
1184- yield [w]*c
1185-
1186+ count = map(int, self.store_count.split())
1187+ for c, w in zip(count, self.store_wares.all()):
1188+ yield [w] * c
1189
1190 def __unicode__(self):
1191- return u"%s/%s" %(self.tribe.name,self.name)
1192-
1193+ return u"%s/%s" % (self.tribe.name, self.name)
1194
1195=== modified file 'wlhelp/tests.py'
1196--- wlhelp/tests.py 2009-02-26 22:38:49 +0000
1197+++ wlhelp/tests.py 2016-05-22 08:48:29 +0000
1198@@ -1,23 +1,24 @@
1199-"""
1200-This file demonstrates two different styles of tests (one doctest and one
1201+"""This file demonstrates two different styles of tests (one doctest and one
1202 unittest). These will both pass when you run "manage.py test".
1203
1204 Replace these with more appropriate tests for your application.
1205+
1206 """
1207
1208 from django.test import TestCase
1209
1210+
1211 class SimpleTest(TestCase):
1212+
1213 def test_basic_addition(self):
1214 """
1215 Tests that 1 + 1 always equals 2.
1216 """
1217 self.failUnlessEqual(1 + 1, 2)
1218
1219-__test__ = {"doctest": """
1220+__test__ = {'doctest': """
1221 Another way to test that 1 + 1 is equal to 2.
1222
1223 >>> 1 + 1 == 2
1224 True
1225 """}
1226-
1227
1228=== modified file 'wlhelp/urls.py'
1229--- wlhelp/urls.py 2012-05-18 23:41:26 +0000
1230+++ wlhelp/urls.py 2016-05-22 08:48:29 +0000
1231@@ -10,18 +10,25 @@
1232 #
1233
1234 from django.conf.urls.defaults import *
1235-from views import *
1236-
1237-urlpatterns= patterns('',
1238- url(r'^$', index, name="wlhelp_index"),
1239-
1240- # Detail pages
1241- url(r'^(?P<tribe>\w+)/$', tribe_details, name="wlhelp_tribe_details"),
1242- url(r'^(?P<tribe>\w+)/wares/(?P<ware>[^/]+)/$', ware_details, name="wlhelp_ware_details"),
1243- url(r'^(?P<tribe>\w+)/buildings/(?P<building>[^/]+)/$', building_details, name="wlhelp_building_details"),
1244- url(r'^(?P<tribe>\w+)/workers/(?P<worker>[^/]+)/$', worker_details, name="wlhelp_worker_details"),
1245-
1246- url(r'^(?P<tribe>\w+)/workers/$', workers, name="wlhelp_workers"),
1247- url(r'^(?P<tribe>\w+)/wares/$', wares, name="wlhelp_wares"),
1248- url(r'^(?P<tribe>\w+)/buildings/$', buildings, name="wlhelp_buildings"),
1249-)
1250+from views import *
1251+
1252+urlpatterns = patterns('',
1253+ url(r'^$', index, name='wlhelp_index'),
1254+
1255+ # Detail pages
1256+ url(r'^(?P<tribe>\w+)/$', tribe_details,
1257+ name='wlhelp_tribe_details'),
1258+ url(r'^(?P<tribe>\w+)/wares/(?P<ware>[^/]+)/$',
1259+ ware_details, name='wlhelp_ware_details'),
1260+ url(r'^(?P<tribe>\w+)/buildings/(?P<building>[^/]+)/$',
1261+ building_details, name='wlhelp_building_details'),
1262+ url(r'^(?P<tribe>\w+)/workers/(?P<worker>[^/]+)/$',
1263+ worker_details, name='wlhelp_worker_details'),
1264+
1265+ url(r'^(?P<tribe>\w+)/workers/$',
1266+ workers, name='wlhelp_workers'),
1267+ url(r'^(?P<tribe>\w+)/wares/$',
1268+ wares, name='wlhelp_wares'),
1269+ url(r'^(?P<tribe>\w+)/buildings/$',
1270+ buildings, name='wlhelp_buildings'),
1271+ )
1272
1273=== modified file 'wlhelp/views.py'
1274--- wlhelp/views.py 2013-06-14 19:23:53 +0000
1275+++ wlhelp/views.py 2016-05-22 08:48:29 +0000
1276@@ -4,94 +4,99 @@
1277 from .models import Worker, Ware, Building, Tribe
1278
1279 from settings import WIDELANDS_SVN_DIR, MEDIA_ROOT
1280-import os
1281-
1282-def index( request ):
1283- tribes = Tribe.objects.all().order_by("displayname")
1284+
1285+
1286+def index(request):
1287+ tribes = Tribe.objects.all().order_by('displayname')
1288
1289 return render_to_response('wlhelp/index.html',
1290- context_instance=RequestContext(request,
1291- { "tribes": tribes }))
1292-
1293-def tribe_details( request, tribe ):
1294+ context_instance=RequestContext(request,
1295+ {'tribes': tribes}))
1296+
1297+
1298+def tribe_details(request, tribe):
1299 t = get_object_or_404(Tribe, name=tribe)
1300
1301 return render_to_response('wlhelp/tribe_details.html',
1302- context_instance=RequestContext(request,
1303- { "tribe": t }))
1304-
1305-def ware_details( request, tribe, ware ):
1306- w = get_object_or_404(Ware,tribe__name=tribe,name=ware)
1307+ context_instance=RequestContext(request,
1308+ {'tribe': t}))
1309+
1310+
1311+def ware_details(request, tribe, ware):
1312+ w = get_object_or_404(Ware, tribe__name=tribe, name=ware)
1313 t = Tribe.objects.get(name=tribe)
1314
1315 return render_to_response('wlhelp/ware_details.html',
1316- context_instance=RequestContext(request,
1317- { "ware": w, "tribe": t }))
1318-
1319-def building_details( request, tribe, building ):
1320- b = get_object_or_404(Building,tribe__name=tribe,name=building)
1321+ context_instance=RequestContext(request,
1322+ {'ware': w, 'tribe': t}))
1323+
1324+
1325+def building_details(request, tribe, building):
1326+ b = get_object_or_404(Building, tribe__name=tribe, name=building)
1327 t = Tribe.objects.get(name=tribe)
1328
1329 return render_to_response('wlhelp/building_details.html',
1330- context_instance=RequestContext(request,
1331- { "building": b, "tribe": t }))
1332-
1333-def worker_details( request, tribe, worker ):
1334- w = get_object_or_404(Worker,tribe__name=tribe,name=worker)
1335+ context_instance=RequestContext(request,
1336+ {'building': b, 'tribe': t}))
1337+
1338+
1339+def worker_details(request, tribe, worker):
1340+ w = get_object_or_404(Worker, tribe__name=tribe, name=worker)
1341 t = Tribe.objects.get(name=tribe)
1342
1343 return render_to_response('wlhelp/worker_details.html',
1344- context_instance=RequestContext(request,
1345- { "worker": w, "tribe": t }))
1346-
1347-def workers(request, tribe="barbarians"):
1348- t = get_object_or_404(Tribe,name=tribe)
1349+ context_instance=RequestContext(request,
1350+ {'worker': w, 'tribe': t}))
1351+
1352+
1353+def workers(request, tribe='barbarians'):
1354+ t = get_object_or_404(Tribe, name=tribe)
1355 return render_to_response('wlhelp/workers.html', context_instance=RequestContext(request,
1356- { "workers": Worker.objects.filter(tribe=t).order_by("displayname"),
1357- "tribe": t }))
1358-
1359-def wares(request, tribe="barbarians"):
1360- t = get_object_or_404(Tribe,name=tribe)
1361+ {'workers': Worker.objects.filter(tribe=t).order_by('displayname'),
1362+ 'tribe': t}))
1363+
1364+
1365+def wares(request, tribe='barbarians'):
1366+ t = get_object_or_404(Tribe, name=tribe)
1367 return render_to_response('wlhelp/wares.html', context_instance=RequestContext(request,
1368- { "wares": Ware.objects.filter(tribe=t).order_by("displayname"),
1369- "tribe": t }))
1370-
1371-def buildings(request, tribe="barbarians"):
1372- t = get_object_or_404(Tribe,name=tribe)
1373+ {'wares': Ware.objects.filter(tribe=t).order_by('displayname'),
1374+ 'tribe': t}))
1375+
1376+
1377+def buildings(request, tribe='barbarians'):
1378+ t = get_object_or_404(Tribe, name=tribe)
1379
1380 # Request all the objects
1381 buildings = {}
1382
1383- buildings["headquarters"] = Building.objects.filter(tribe=t,name="headquarters").order_by("displayname")
1384-
1385- all = Building.objects.filter(tribe=t).exclude(name="headquarters")
1386+ # All headquarters
1387+ buildings['headquarters'] = Building.objects.filter(
1388+ size='H', tribe=t).order_by('displayname')
1389
1390 # Now, all small buildings
1391- small = all.filter(size="S",tribe=t).order_by("displayname")
1392- buildings["small"] = small.filter(enhanced_from=None)
1393- buildings["small_enhanced"] = small.exclude(enhanced_from=None)
1394+ small = Building.objects.filter(size='S', tribe=t).order_by('displayname')
1395+ buildings['small'] = small.filter(enhanced_from=None)
1396+ buildings['small_enhanced'] = small.exclude(enhanced_from=None)
1397
1398 # Now, all medium buildings
1399- medium = all.filter(size="M",tribe=t).order_by("displayname")
1400- buildings["medium"] = medium.filter(enhanced_from=None)
1401- buildings["medium_enhanced"] = medium.exclude(enhanced_from=None)
1402+ medium = Building.objects.filter(size='M', tribe=t).order_by('displayname')
1403+ buildings['medium'] = medium.filter(enhanced_from=None)
1404+ buildings['medium_enhanced'] = medium.exclude(enhanced_from=None)
1405
1406 # Now, all big buildings
1407- big = all.filter(size="B",tribe=t).order_by("displayname")
1408- buildings["big"] = big.filter(enhanced_from=None)
1409- buildings["big_enhanced"] = big.exclude(enhanced_from=None)
1410+ big = Building.objects.filter(size='B', tribe=t).order_by('displayname')
1411+ buildings['big'] = big.filter(enhanced_from=None)
1412+ buildings['big_enhanced'] = big.exclude(enhanced_from=None)
1413
1414 # Now, all mines
1415- mine = all.filter(size="I",tribe=t).order_by("displayname")
1416- buildings["mine"] = mine.filter(enhanced_from=None)
1417- buildings["mine_enhanced"] = mine.exclude(enhanced_from=None)
1418+ mine = Building.objects.filter(size='I', tribe=t).order_by('displayname')
1419+ buildings['mine'] = mine.filter(enhanced_from=None)
1420+ buildings['mine_enhanced'] = mine.exclude(enhanced_from=None)
1421
1422 # Now, all ports
1423- port = all.filter(size="P",tribe=t).order_by("displayname")
1424- buildings["port"] = port.filter(enhanced_from=None)
1425- buildings["port_enhanced"] = port.exclude(enhanced_from=None)
1426+ port = Building.objects.filter(size='P', tribe=t).order_by('displayname')
1427+ buildings['port'] = port.filter(enhanced_from=None)
1428+ buildings['port_enhanced'] = port.exclude(enhanced_from=None)
1429
1430 return render_to_response('wlhelp/buildings.html', context_instance=RequestContext(request,
1431- { "buildings": buildings, "tribe": t }))
1432-
1433-
1434+ {'buildings': buildings, 'tribe': t}))

Subscribers

People subscribed via source and target branches