Merge lp:~mgiuca/mugle/gamefiles into lp:mugle

[Matt Giuca]

If anyone is still up, can you have a look at this diff. It introduces game file uploading. I just want to check if I am doing the database access right -- particularly with the definition of the GameFileContents class, and the way in which GameFileData and GameFileContents objects are created and looked up.

If not, I'll just commit it.

Note that I'm not quite ready to commit; I need to add MIME type detection and game file serving, but that doesn't need serious review.

[lp:~mgiuca/mugle/gamefiles]

324. By Matt Giuca on 2011-05-22

Merge from trunk.

325. By Matt Giuca on 2011-05-22

UploadService: As Scott recommends, open a new PersistenceManager for each file.

326. By Matt Giuca on 2011-05-22

GameFileData: Now correctly sets the MIME type of a file based on its extension.
Added src/META-INF/mime.types, a table mapping file extensions to MIME types (manually created).

327. By Matt Giuca on 2011-05-22

web.xml: Set up URL handler for GameFileServer.
Unfortunately, had to change URL pattern to /+games/* since I can't match /*/*/+play in web.xml, as originally planned.

328. By Matt Giuca on 2011-05-22

GameFileServer: Started handling URL.

329. By Matt Giuca on 2011-05-22

GameFileServer: Clean up debugging output.

330. By Matt Giuca on 2011-05-22

GameVersionGetter: Fix getGameVersion by Key double-closing the persistence manager.

331. By Matt Giuca on 2011-05-22

GameVersionGetter: More useful methods.

332. By Matt Giuca on 2011-05-22

GameGetter: getGameByDevTeam now actually takes a game name.
What ... was it doing before? It seemed to be expecting that each devteam had only one game, and didn't check the name.

333. By Matt Giuca on 2011-05-22

GameFileServer: Now gets the gameversion associated with the URL, and throws errors if they are missing.

334. By Matt Giuca on 2011-05-22

GameFileGetter: More methods.

335. By Matt Giuca on 2011-05-22

Fix GameFileGetter error messages.

336. By Matt Giuca on 2011-05-22

GameFileGetter: Can now get GameFileContents from a GameFile.
Added new exception type GameFileContentsNotExists.

337. By Matt Giuca on 2011-05-22

UploadService now uses GameFileGetter.getGameFileContents.

338. By Matt Giuca on 2011-05-22

GameFileServer now gets the game file and contents specified.

339. By Matt Giuca on 2011-05-22

GameFileContents: getContents now works.

340. By Matt Giuca on 2011-05-22

GameFileServer: Prints the MIME type and contents to the debugging output.

341. By Matt Giuca on 2011-05-22

GameFileServer: Replaced debug output with actual file service.
Now serves the raw bytes of the file with the appropriate MIME type.
Looks good ... up until around 0x200 bytes where it becomes all 0s.

342. By Matt Giuca on 2011-05-22

GameFileServer: Now sets Content-Length for better browser handling.

343. By Matt Giuca on 2011-05-22

GameFileContents: Fix bug where unzipping files beyond a certain size would truncate the file with null bytes.
It needs to read in a loop, because stream.read won't necessarily return all the bytes at once.

344. By Matt Giuca on 2011-05-22

UploadService: Fix bug where uploading a new file over an old one would not replace the contents.
For some reason you have to call pm.makePersistent on contents. I thought you didn't need to if the object already existed?
(It only is for this object; the other ones you don't need to call makePersistent to make a change. Strange.)

345. By Matt Giuca on 2011-05-22

UploadService: Change result so it's no longer debug text.
It shows the list of filenames -- it'll do for now as a user interface.

346. By Matt Giuca on 2011-05-22

UploadService: Remove the first path segment from the filename, so the top-level directory of the zip file is ignored.

347. By Matt Giuca on 2011-05-22

GameEditBuilder: Added instructions for uploading a zip file.

=== modified file 'doc/platform/urls.rst'
--- doc/platform/urls.rst	2011-05-22 08:01:47 +0000
+++ doc/platform/urls.rst	2011-05-22 18:34:23 +0000
@@ -82,13 +82,14 @@
     /                   Alias for /Mugle.html
     /Mugle.html         The main web page
     /mugle/             Contains static JavaScript and Ajax URLs
+        /upload         Game upload target
     /img/               Contains static images
     /favicon.ico
     /gwt-override.css
     /Mugle.css
     /MugleIE6.css
-    /<promoted-game>/+play/     Static game files for game
-    /<team>/<game>/+play/       Static game files for game
+    /+games/<promoted-game>/    Static game files for game
+    /+games/<team>/<game>/      Static game files for game
 
 Client URLs
 -----------

=== added file 'src/META-INF/mime.types'
--- src/META-INF/mime.types	1970-01-01 00:00:00 +0000
+++ src/META-INF/mime.types	2011-05-22 18:34:23 +0000
@@ -0,0 +1,35 @@
+# MUGLE MIME types mapping
+# Used to set the MIME type of files being uploaded
+# This list should be just about all the file types students will need.
+# the format is <mime type> <space separated file extensions>
+
+# Application
+application/json json
+application/javascript js
+application/pdf pdf
+application/xhtml+xml xhtml
+application/zip zip
+application/x-gzip gz gzip
+
+# Text
+text/plain txt text
+text/html htm html
+text/css css
+text/xml xml
+
+# Image
+image/gif gif
+image/jpeg jpg jpeg
+image/png png
+image/svg svg
+image/webp webp
+
+# Audio
+audio/mpeg mp3
+audio/ogg ogg
+audio/vnd.wave wav
+
+# Video
+video/mpeg mpeg
+video/mp4 mp4
+video/webm webm

=== modified file 'src/au/edu/unimelb/csse/mugle/client/ui/GameEditBuilder.java'
--- src/au/edu/unimelb/csse/mugle/client/ui/GameEditBuilder.java	2011-05-22 13:33:19 +0000
+++ src/au/edu/unimelb/csse/mugle/client/ui/GameEditBuilder.java	2011-05-22 18:34:23 +0000
@@ -16,6 +16,10 @@
 import com.google.gwt.user.client.rpc.AsyncCallback;
 import com.google.gwt.user.client.ui.*;
 import com.google.gwt.user.client.ui.HTMLTable.RowFormatter;
+import com.google.gwt.user.client.ui.FormPanel;
+import com.google.gwt.user.client.ui.FileUpload;
+
+import com.google.gwt.user.client.Window;
 
 /**
  * Loads the view for editing game information.
@@ -105,6 +109,7 @@
         panel.add(warning);
         panel.add(new Label("Badges"));
         panel.add(assembleAchievementsTable());
+        panel.add(assembleUploadBox(gameVersion));
 
         // Assemble namePanel
         namePanel.add(new Label("Name: "));
@@ -452,6 +457,55 @@
             }           
         });
     }
+
+    /**
+     * Creates a widget allowing the user to upload a new game in a zip file.
+     * @param version Game version to upload over the top of.
+     */
+    private Widget assembleUploadBox(GameVersion version) {
+        final FormPanel form = new FormPanel();
+        form.setAction("/mugle/upload?gameversion="
+            + Long.toString(version.getPrimaryKey()));
+
+        // File upload widget requires multipart/form-data encoding, and POST
+        // method.
+        form.setEncoding(FormPanel.ENCODING_MULTIPART);
+        form.setMethod(FormPanel.METHOD_POST);
+
+        VerticalPanel vpanel = new VerticalPanel();
+        form.setWidget(vpanel);
+        vpanel.add(new Label("Upload the game code. Instructions: Compile "
+        + "your GWT project and then zip up the 'war' directory. Your zip "
+        + "file must have a single top-level directory. Upload the zip file "
+        + "here. This will replace any existing versions of the game."));
+
+        HorizontalPanel hpanel = new HorizontalPanel();
+        vpanel.add(hpanel);
+
+        // Create a FileUpload widget
+        FileUpload upload = new FileUpload();
+        upload.setName("game_upload");
+        hpanel.add(upload);
+
+        // Add a 'submit' button
+        hpanel.add(new Button("Upload", new ClickListener() {
+          public void onClick(Widget sender) {
+            form.submit();
+          }
+        }));
+
+
+        // Handle the response from the server after upload
+        form.addFormHandler(new FormHandler() {
+          public void onSubmit(FormSubmitEvent event) {}
+
+          public void onSubmitComplete(FormSubmitCompleteEvent event) {
+            Window.alert(event.getResults());
+          }
+        });
+
+        return form;
+    }
     
     /**
      * Constructs the Navigation bar down the bottom (with delete button)

=== modified file 'src/au/edu/unimelb/csse/mugle/server/DataTestServiceImpl.java'
--- src/au/edu/unimelb/csse/mugle/server/DataTestServiceImpl.java	2011-05-22 13:10:42 +0000
+++ src/au/edu/unimelb/csse/mugle/server/DataTestServiceImpl.java	2011-05-22 18:34:23 +0000
@@ -100,7 +100,6 @@
 
       gamefiles[0].setPath("index.html");
       gamefiles[0].setMimeType("text/html");
-      gamefiles[0].setBlobKey(null);        // XXX
       gamefiles[0].setGameVersionKey(gameversions[0].getPrimaryKey());
 
       GameFileServiceImpl gfs = new GameFileServiceImpl();

=== modified file 'src/au/edu/unimelb/csse/mugle/server/GameFileServer.java'
--- src/au/edu/unimelb/csse/mugle/server/GameFileServer.java	2011-05-20 06:35:39 +0000
+++ src/au/edu/unimelb/csse/mugle/server/GameFileServer.java	2011-05-22 18:34:23 +0000
@@ -16,37 +16,92 @@
  */
 package au.edu.unimelb.csse.mugle.server;
 
+import java.io.OutputStream;
 import java.io.IOException;
 
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 
+import com.google.appengine.api.datastore.Key;
+
+import au.edu.unimelb.csse.mugle.server.model.GameData;
+import au.edu.unimelb.csse.mugle.server.model.GameGetter;
+import au.edu.unimelb.csse.mugle.server.model.GameVersionData;
+import au.edu.unimelb.csse.mugle.server.model.GameVersionGetter;
 import au.edu.unimelb.csse.mugle.server.model.GameFileData;
 import au.edu.unimelb.csse.mugle.server.model.GameFileGetter;
+import au.edu.unimelb.csse.mugle.server.model.GameFileContents;
+import au.edu.unimelb.csse.mugle.shared.platform.exceptions.DevTeamNotExists;
 import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameFileNotExists;
+import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameFileContentsNotExists;
 import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameNotExists;
 import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameVersionNotExists;
 
-import com.google.appengine.api.blobstore.BlobKey;
-import com.google.appengine.api.blobstore.BlobstoreService;
-import com.google.appengine.api.blobstore.BlobstoreServiceFactory;
-
 @SuppressWarnings("serial")
 public class GameFileServer extends HttpServlet {
     
     public void doGet(HttpServletRequest req, HttpServletResponse res) 
         throws IOException {
-        BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
-        
-        try {
-            BlobKey blobKey = getBlobKey(req);
-            blobstoreService.serve(blobKey, res);
-        } catch (GameFileNotExists e) {
-            //TODO: Construct a page with the error
-        }
+        String path = req.getPathInfo();
+        if (path.length() > 0 && path.charAt(0) == '/')
+            path = path.substring(1);
+        String[] pathSegments = path.split("/", 3);
+        // XXX Assumes a devteam/game, not a promoted game
+        if (pathSegments.length != 3)
+        {
+            res.sendError(404, "Not found");
+            return;
+        }
+        String devteamName = pathSegments[0];
+        String gameName = pathSegments[1];
+        String filePath = pathSegments[2];
+
+        GameFileData file;
+        GameFileContents contents;
+        try
+        {
+            GameData game = GameGetter.getGameByDevTeam(devteamName,gameName);
+            Key versionKey = game.getActiveVersion();
+            GameVersionData version =
+                GameVersionGetter.getGameVersion(versionKey);
+            file = GameFileGetter.getGameFile(game, version, filePath);
+            contents = GameFileGetter.getGameFileContents(file);
+        }
+        catch (DevTeamNotExists e)
+        {
+            res.sendError(404, e.toString());
+            return;
+        }
+        catch (GameNotExists e)
+        {
+            res.sendError(404, e.toString());
+            return;
+        }
+        catch (GameVersionNotExists e)
+        {
+            res.sendError(404, e.toString());
+            return;
+        }
+        catch (GameFileNotExists e)
+        {
+            res.sendError(404, e.toString());
+            return;
+        }
+        catch (GameFileContentsNotExists e)
+        {
+            res.sendError(404, e.toString());
+            return;
+        }
+
+        // Serve the file on the HTTP response
+        res.setContentType(file.getMimeType());
+        byte[] bytes = contents.getContents();
+        res.setContentLength(bytes.length);
+        res.getOutputStream().write(bytes);
     }
 
+    /*
     private BlobKey getBlobKey(HttpServletRequest req) 
         throws GameFileNotExists {
         
@@ -57,7 +112,7 @@
         /* We need to parse the request to get the gameName, gameVersion
          * and path,
          * URLs are in the form /game/gameName/gameVersion/path
-         */
+         * /
         String toParse = req.getRequestURI().split("/game/")[1];  // gets everything after /game/ in the URI
         String[] splitted = toParse.split("/");
         gameName = splitted[0];
@@ -74,6 +129,7 @@
         }
         
     }
+    */
 
     /*
     public void doPost(HttpServletRequest req, HttpServletResponse res) 

=== added file 'src/au/edu/unimelb/csse/mugle/server/UploadService.java'
--- src/au/edu/unimelb/csse/mugle/server/UploadService.java	1970-01-01 00:00:00 +0000
+++ src/au/edu/unimelb/csse/mugle/server/UploadService.java	2011-05-22 18:34:23 +0000
@@ -0,0 +1,197 @@
+/*  Melbourne University Game-based Learning Environment
+ *  Copyright (C) 2011 The University of Melbourne
+ *
+ *  This program is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+package au.edu.unimelb.csse.mugle.server;
+
+import java.io.InputStream;
+import java.io.PrintWriter;
+import java.io.IOException;
+import javax.servlet.http.HttpServlet;
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import javax.jdo.PersistenceManager;
+
+import java.util.zip.ZipInputStream;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipException;
+
+import org.apache.commons.fileupload.FileItemIterator;
+import org.apache.commons.fileupload.FileItemStream;
+import org.apache.commons.fileupload.servlet.ServletFileUpload;
+import org.apache.commons.fileupload.FileUploadException;
+
+import java.util.Vector;
+
+import au.edu.unimelb.csse.mugle.server.model.GameFileData;
+import au.edu.unimelb.csse.mugle.server.model.GameFileContents;
+import au.edu.unimelb.csse.mugle.server.model.GameFileGetter;
+import au.edu.unimelb.csse.mugle.server.model.GameVersionData;
+import au.edu.unimelb.csse.mugle.server.PMF;
+import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameFileNotExists;
+import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameFileContentsNotExists;
+
+public class UploadService extends HttpServlet
+{
+    public void doPost(HttpServletRequest request,
+        HttpServletResponse response) throws IOException
+    {
+        long gameversionID = Long.parseLong(
+                                request.getParameter("gameversion"));
+        GameVersionData gameversion;
+        boolean got_zip = false;
+        int numfiles = 0;
+        Vector<String> filenames = new Vector<String>();
+
+        // Read the POST data fields
+        ServletFileUpload upload = new ServletFileUpload();
+        PersistenceManager pm = PMF.getManager();
+        try
+        {
+            // Get the GameVersion object
+            gameversion =
+                pm.getObjectById(GameVersionData.class, gameversionID);
+        }
+        finally
+        {
+            pm.close();
+        }
+        try
+        {
+            // For each field
+            FileItemIterator iter = upload.getItemIterator(request);
+            while (iter.hasNext())
+            {
+                FileItemStream item = iter.next();
+                // Only care about the game_upload field
+                if (!item.getFieldName().equals("game_upload"))
+                    continue;
+                got_zip = true;
+
+                // Assume it is a ZIP file. Read each file contained inside.
+                InputStream stream = item.openStream();
+                ZipInputStream zstream = new ZipInputStream(stream);
+                ZipEntry entry;
+                while ((entry = zstream.getNextEntry()) != null)
+                {
+                    if (entry.isDirectory())
+                    {
+                        // Ignore directories (since we are not building a
+                        // tree; just remembering the path to each file)
+                        continue;
+                    }
+                    String filename = entry.getName();
+                    // Remove the first path segment from the filename (since
+                    // ZIP files tend to have a top-level directory, and we
+                    // want to explode out of that).
+                    String[] filename_split = filename.split("/", 2);
+                    if (filename_split.length > 1)
+                        filename = filename_split[1];
+                    long size = entry.getSize();
+                    // Create a new GameFile object
+                    createGameFile(gameversion, filename, zstream, size);
+                    filenames.add(filename);
+                    numfiles++;
+                }
+            }
+        }
+        catch (FileUploadException e)
+        {
+            response.sendError(400, e.toString());
+            return;
+        }
+        catch (ZipException e)
+        {
+            response.sendError(400, e.toString());
+            return;
+        }
+        if (!got_zip)
+        {
+            response.sendError(400, "Missing POST field game_upload");
+            return;
+        }
+
+        // Set HTML content type, even though returning text
+        // The result is shown in an alert, and if it's text, Firefox will add
+        // "<pre>".
+        response.setContentType("text/html");
+        PrintWriter out = response.getWriter();
+        out.println("Successfully uploaded a ZIP file with "
+            + numfiles + " files:");
+        for (String s : filenames)
+            out.println("- " + s);
+    }
+
+    /** Create a new GameFile and store it in the database, from a part of a
+     * Zip file.
+     * @param gameVersionKey The game version to upload this file to.
+     * @param filePath Path to the file relative to this game version.
+     * @param data Binary stream to read file contents from.
+     * @param dataSize Number of bytes to read from data.
+     */
+    public static void createGameFile(GameVersionData gameVersion,
+        String filePath, InputStream data, long dataSize)
+        throws IOException
+    {
+        PersistenceManager pm = PMF.getManager();
+        try
+        {
+            // Check if a file already exists at that path
+            GameFileData file;
+            GameFileContents contents;
+            try
+            {
+                file = GameFileGetter.getGameFile(pm, gameVersion, filePath);
+            }
+            catch (GameFileNotExists e)
+            {
+                // First, store the contents as a blob in the database
+                contents = new GameFileContents(data, dataSize);
+                pm.makePersistent(contents);
+                // Then, create a GameFile which points at that blob
+                file = new GameFileData(gameVersion, filePath, contents);
+                pm.makePersistent(file);
+                return;
+            }
+            // Already exists; just update the file and contents
+            // (Note that if we were uploading a new version, we wouldn't
+            // overwrite the file or contents, but since we're updating an
+            // existing version, we can replace the contents).
+            // XXX Note that if contents are shared across versions this will
+            // clobber ALL versions, so probably we should create a new
+            // content object each time.
+            try
+            {
+                contents = GameFileGetter.getGameFileContents(file);
+                contents.setContents(data, dataSize);
+                pm.makePersistent(contents);
+            }
+            catch (GameFileContentsNotExists e)
+            {
+                contents = new GameFileContents(data, dataSize);
+                pm.makePersistent(contents);
+                file.setContents(contents.getServerKey());
+            }
+            // Also set the MIME type based on the filename (so the result is
+            // consistent with what would have happened if the file didn't
+            // exist).
+            file.setMimeTypeFromFilename();
+        }
+        finally
+        {
+            pm.close();
+        }
+    }
+}

=== added file 'src/au/edu/unimelb/csse/mugle/server/model/GameFileContents.java'
--- src/au/edu/unimelb/csse/mugle/server/model/GameFileContents.java	1970-01-01 00:00:00 +0000
+++ src/au/edu/unimelb/csse/mugle/server/model/GameFileContents.java	2011-05-22 18:34:23 +0000
@@ -0,0 +1,104 @@
+/*  Melbourne University Game-based Learning Environment
+ *  Copyright (C) 2011 The University of Melbourne
+ *
+ *  This program is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package au.edu.unimelb.csse.mugle.server.model;
+
+import java.io.OutputStream;
+import java.io.InputStream;
+import java.io.IOException;
+
+import javax.jdo.PersistenceManager;
+import javax.jdo.annotations.IdGeneratorStrategy;
+import javax.jdo.annotations.PersistenceCapable;
+import javax.jdo.annotations.Persistent;
+import javax.jdo.annotations.PrimaryKey;
+
+import com.google.appengine.api.datastore.Key;
+import com.google.appengine.api.datastore.Blob;
+
+/** The contents of a GameFile.
+ * This simple class contains only the contents and no other meta-data. It
+ * may be referenced by one or more GameFile objects.
+ */
+@PersistenceCapable
+public class GameFileContents {
+    // Fields
+
+    @PrimaryKey
+    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
+    private Key id;
+
+    @Persistent
+    private Blob data;
+
+    /** Create a new file contents blob, from a given input stream.
+     @param size The number of bytes to read from stream.
+     */
+    public GameFileContents(InputStream stream, long size)
+        throws IOException
+    {
+        this.setContents(stream, size);
+    }
+
+    // Getters
+
+    public Key getServerKey() {
+        return this.id;
+    }
+
+    // Misc methods
+
+    /** Get the contents of the file as a byte array.
+     */
+    public byte[] getContents()
+    {
+        return this.data.getBytes();
+    }
+
+    /** Write the contents of the file to the OutputStream object.
+     * @param writer Stream object to receive the contents of the file.
+     */
+    public void getContents(OutputStream writer)
+        throws IOException
+    {
+        byte[] bytes = this.data.getBytes();
+        writer.write(bytes);
+    }
+
+    /** Read the contents of an InputStream and store it in the database as
+     * the contents of this file.
+     * @param stream Stream object to read contents of file from.
+     @param size The number of bytes to read from stream.
+     */
+    public void setContents(InputStream stream, long size)
+        throws IOException
+    {
+        //TODO: check that the DevTeam's space limit hasn't been exceeded.
+        byte[] bytes = new byte[(int)size];
+        // Read in a loop
+        // (stream.read might read fewer than the requested number of bytes)
+        int offset = 0;
+        int remaining = (int)size;
+        while (remaining > 0)
+        {
+            int bytes_read = stream.read(bytes, offset, remaining);
+            offset += bytes_read;
+            remaining -= bytes_read;
+        }
+        this.data = new Blob(bytes);
+    }
+}

=== modified file 'src/au/edu/unimelb/csse/mugle/server/model/GameFileData.java'
--- src/au/edu/unimelb/csse/mugle/server/model/GameFileData.java	2011-05-22 11:06:46 +0000
+++ src/au/edu/unimelb/csse/mugle/server/model/GameFileData.java	2011-05-22 18:34:23 +0000
@@ -24,9 +24,10 @@
 import javax.jdo.annotations.PersistenceCapable;
 import javax.jdo.annotations.Persistent;
 import javax.jdo.annotations.PrimaryKey;
+import javax.activation.MimetypesFileTypeMap;
 
-import com.google.appengine.api.blobstore.BlobKey;
 import com.google.appengine.api.datastore.Key;
+import com.google.appengine.api.datastore.Blob;
 
 import au.edu.unimelb.csse.mugle.shared.model.*;
 import au.edu.unimelb.csse.mugle.server.PMF;
@@ -50,7 +51,7 @@
     private String mimeType; //mime type of the file
 
     @Persistent
-    private BlobKey blobKey; //key to the actual blob -- may be shared between GameFiles across versions
+    private Key contents; //key to the GameFileContents
 
     @Persistent
     private Key version; //Game version this Gamefile belongs to
@@ -72,23 +73,43 @@
     public GameFileData() {
         this.version = null;
         this.setMimeType(null);
-        this.setBlobKey(null);
         this.path = null;
     }
 
-/** TODO: Pending Blob implementation for files
-    // For creation of the file for the first time
-    public GameFile (String path, GameVersion version, InputStream contents) {
-        this.path = path;
-        this.versions.add(version.getPrimaryKey());
-        this.setContents(contents);
-    }
-*/
+    /** Create a new GameFileData object.
+     * Guesses the file's MIME type based on the file extension.
+     * @param version The game version the file belongs to.
+     * @param path Path to the file relative to this gameversion.
+     * @param contents Blob containing the file's contents.
+     */
+    public GameFileData(GameVersionData version, String path,
+        GameFileContents contents)
+    {
+        this.version = version.getServerKey();
+        this.path = path;
+        this.setMimeTypeFromFilename();
+        this.contents = contents.getServerKey();
+    }
+
+    /** Create a new GameFileData object.
+     * @param version The game version the file belongs to.
+     * @param path Path to the file relative to this gameversion.
+     * @param mimeType MIME type of the file.
+     * @param contents Blob containing the file's contents.
+     */
+    public GameFileData(GameVersionData version, String path, String mimeType,
+        GameFileContents contents)
+    {
+        this.version = version.getServerKey();
+        this.path = path;
+        this.mimeType = mimeType;
+        this.contents = contents.getServerKey();
+    }
 
     /* When a file is moved, or renamed - The existing GameFile entry should be deleted,
      * in which case, we may want to initialise with multiple Game versions.
      */
-/**  TODO: Pending Blob implementation for files
+/**  TODO: Pending multiple versions of games
     public GameFile (String path, Set<String> versions, InputStream contents) {
         this.path = path;
         this.versions = versions;
@@ -122,18 +143,6 @@
         return mimeType;
     }
 
-    @GetterUserLevel(privateView=Role.GUEST, publicView=Role.GUEST, mappedTo="setBlobKey", overrideBy="getBlobKeyString")
-    public BlobKey getBlobKey() {
-        return this.blobKey;
-    }
-
-    public String getBlobKeyString() {
-        if (this.blobKey == null) {
-            return null;
-        }
-        return this.blobKey.getKeyString();
-    }
-
     @Override
     @GetterUserLevel(privateView=Role.GUEST, publicView=Role.GUEST, mappedTo="setCreatedUser", overrideBy="keyToCreated")
     public Key getCreatedUser() {
@@ -209,40 +218,30 @@
         this.mimeType = mimeType;
     }
 
-    // TODO: do we allow the clients to modify this value
-    //@SetterUserLevel(getter="", mappedBy="")
-    public void setBlobKey(BlobKey blobKey) {
-        this.blobKey = blobKey;
-    }
-
     // Misc methods
 
-    /** Write the contents of the file to the PrintWriter object.
-     * @param writer Stream object to receive the contents of the file.
-     */
-/**  TODO: Pending Blob implementation for files
-    public void getContents(PrintWriter writer) {
-        //TODO: retrieve file from database
-    }
-*/
-
-    /** Read the contents of an InputStream and store it in the database as
-     * the contents of this file.
-     * @param stream Stream object to read contents of file from.
-     */
-/**  TODO: Pending Blob implementation for files
-    public void setContents(InputStream stream) {
-        //TODO: add/create the file in the database
-        //TODO: check that the DevTeam's space limit hasn't been exceeded.
-    }
-*/
-
-
-/**  TODO: Pending Blob implementation for files
-    public GameFileData(String path, GameVersion version, InputStream contents) {
-        super(path, version, contents);
-    }
-*/
+    /** Get the content blob of this file.
+     * @return Key of the GameFileContents object.
+     */
+    public Key getContents() {
+        return this.contents;
+    }
+
+    /** Set the content blob of this file.
+     * @param contents Key of the GameFileContents object.
+     */
+    public void setContents(Key contents) {
+        this.contents = contents;
+    }
+
+    /** Set the MIME type based on the file extension.
+     */
+    public void setMimeTypeFromFilename() {
+        String f = this.path.toLowerCase();
+        // This implicitly looks at the MIME mapping table in
+        // src/META-INF/mime.types.
+        this.mimeType = new MimetypesFileTypeMap().getContentType(this.path);
+    }
 
     @Override
     public Key getServerKey() {

=== modified file 'src/au/edu/unimelb/csse/mugle/server/model/GameFileGetter.java'
--- src/au/edu/unimelb/csse/mugle/server/model/GameFileGetter.java	2011-05-22 11:06:46 +0000
+++ src/au/edu/unimelb/csse/mugle/server/model/GameFileGetter.java	2011-05-22 18:34:23 +0000
@@ -27,6 +27,7 @@
 
 import au.edu.unimelb.csse.mugle.server.PMF;
 import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameFileNotExists;
+import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameFileContentsNotExists;
 import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameNotExists;
 import au.edu.unimelb.csse.mugle.shared.platform.exceptions.GameVersionNotExists;
 
@@ -114,6 +115,27 @@
     
     /**
      * Gets the GameFile by its path, version and game, for editing object in datastore
+     * @param gameName the name of the game
+     * @param gameVersion the version of the game
+     * @param path the path of the file
+     * @return the GameFile - read only
+     * @throws GameNotExists
+     * @throws GameVersionNotExists
+     * @throws GameFileNotExists 
+     */
+    @SuppressWarnings("unchecked")
+    public static GameFileData getGameFile(GameData game, GameVersionData gameVersion, String path)
+        throws GameFileNotExists {
+        PersistenceManager pm = PMF.getManager();
+        try {
+            return getGameFile(pm, game, gameVersion, path);
+        } finally {
+            pm.close();
+        }
+    }
+
+    /**
+     * Gets the GameFile by its path, version and game, for editing object in datastore
      * PersistenceManager must be handled by the caller
      * @para pm The Persistence Manager
      * @param gameName the name of the game
@@ -126,14 +148,15 @@
      */
     @SuppressWarnings("unchecked")
     public static GameFileData getGameFile(PersistenceManager pm, GameData game, GameVersionData gameVersion, String path)
-        throws GameFileNotExists, GameNotExists, GameVersionNotExists {
+        throws GameFileNotExists {
 
             Query q = pm.newQuery(GameFileData.class, "path == p && version == v");
             q.declareParameters("String p, com.google.appengine.api.datastore.Key v");
             List<GameFileData> results = (List<GameFileData>) q.execute(path, gameVersion.getServerKey());
 
             if (results.isEmpty()) {
-                throw new GameFileNotExists(game.getUrlName(), String.valueOf(gameVersion), path);
+                throw new GameFileNotExists(game.getUrlName(),
+                    gameVersion.getDisplayVersion(), path);
             }
             return results.get(0);
     }
@@ -160,13 +183,41 @@
         List<GameFileData> results = (List<GameFileData>) q.execute(path, gvd.getServerKey());
 
         if (results.isEmpty()) {
-            throw new GameFileNotExists(game.getUrlName(), String.valueOf(gameVersion), path);
+            throw new GameFileNotExists(game.getUrlName(),
+                Integer.toString(gameVersion), path);
         }
         
         return results.get(0);
     }
     
     /**
+     * Gets the GameFile by its path and version, for editing object in
+     * datastore.
+     * PersistenceManager must be handled by the caller
+     * @para pm The Persistence Manager
+     * @param gameVersion the version of the game
+     * @param path the path of the file
+     * @return the GameFile - read only
+     * @throws GameFileNotExists
+     */
+    @SuppressWarnings("unchecked")
+    public static GameFileData getGameFile(PersistenceManager pm,
+        GameVersionData gameVersion, String path)
+        throws GameFileNotExists {
+
+        Query q = pm.newQuery(GameFileData.class, "path == p && version == v");
+        q.declareParameters("String p, com.google.appengine.api.datastore.Key v");
+        List<GameFileData> results = (List<GameFileData>) q.execute(path,
+            gameVersion.getServerKey());
+
+        if (results.isEmpty()) {
+            throw new GameFileNotExists(gameVersion.getDisplayVersion(), path);
+        }
+
+        return results.get(0);
+    }
+
+    /**
      * Gets the GameFile associated with the given GameFileData primary key.
      * @param pm The PersistenceManager
      * @param key The server-side key (ModelDataClass.getServerKey())
@@ -187,5 +238,39 @@
        }
        
     }
-    
+
+    /**
+     * Gets the GameFileContents associated with the given GameFile.
+     * @param file The file to retrieve contents for.
+     * @return The GameFileContents object directly off the datastore.
+     * @throws GameFileContentsNotExists
+     */
+    public static GameFileContents getGameFileContents(GameFileData file)
+        throws GameFileContentsNotExists {
+        PersistenceManager pm = PMF.getManager();
+        try {
+            return getGameFileContents(pm, file);
+        } finally {
+            pm.close();
+        }
+    }
+
+    /**
+     * Gets the GameFileContents associated with the given GameFile.
+     * @param pm The PersistenceManager
+     * @param file The file to retrieve contents for.
+     * @return The GameFileContents object directly off the datastore.
+     * @throws GameFileContentsNotExists
+     */
+    public static GameFileContents getGameFileContents(PersistenceManager pm,
+        GameFileData file) throws GameFileContentsNotExists {
+        Key key = file.getContents();
+        GameFileContents result = pm.getObjectById(GameFileContents.class,
+                                                   key);
+        if (result == null)
+        {
+            throw new GameFileContentsNotExists(key.getId());
+        }
+        return result;
+    }
 }

=== modified file 'src/au/edu/unimelb/csse/mugle/server/model/GameGetter.java'
--- src/au/edu/unimelb/csse/mugle/server/model/GameGetter.java	2011-05-20 06:35:39 +0000
+++ src/au/edu/unimelb/csse/mugle/server/model/GameGetter.java	2011-05-22 18:34:23 +0000
@@ -185,15 +185,17 @@
    
     /**
      * Gets the Game by the name of the DevTeam it belongs to - READ ONLY
-     * @param name name of the DevTeam
+     * @param devTeamName name of the DevTeam
+     * @param gameName name of the Game
      * @return the Game - read only
      * @throws GameNotExists 
      * @throws DevTeamNotExists 
      */
-    public static GameData getGameByDevTeam(String name) throws GameNotExists, DevTeamNotExists {
+    public static GameData getGameByDevTeam(String devTeamName,
+        String gameName) throws GameNotExists, DevTeamNotExists {
         PersistenceManager pm = PMF.getManager();
         try {
-            return getGameByDevTeam(pm, name);
+            return getGameByDevTeam(pm, devTeamName, gameName);
         } finally {
             pm.close();
         }
@@ -209,14 +211,17 @@
      * @throws DevTeamNotExists 
      */
     @SuppressWarnings("unchecked")
-    public static GameData getGameByDevTeam(PersistenceManager pm, String name) throws GameNotExists, DevTeamNotExists{
-        DevTeamData dtd = DevTeamGetter.getDevTeam(name);
-        Query q = pm.newQuery(GameData.class, "devTeam == d");
-        q.declareParameters("com.google.appengine.api.datastore.Key d");
-        List<GameData> results = (List<GameData>) q.execute(dtd.getServerKey());
+    public static GameData getGameByDevTeam(PersistenceManager pm,
+        String devTeamName, String gameName)
+        throws GameNotExists, DevTeamNotExists{
+        DevTeamData dtd = DevTeamGetter.getDevTeam(devTeamName);
+        Query q = pm.newQuery(GameData.class, "devTeam == d && urlName == g");
+        q.declareParameters("com.google.appengine.api.datastore.Key d, String g");
+        List<GameData> results = (List<GameData>)
+            q.execute(dtd.getServerKey(), gameName);
         
         if(results.isEmpty()) {
-            throw new GameNotExists(name);
+            throw new GameNotExists(devTeamName, gameName);
         }
         
         return results.get(0);

=== modified file 'src/au/edu/unimelb/csse/mugle/server/model/GameVersionGetter.java'
--- src/au/edu/unimelb/csse/mugle/server/model/GameVersionGetter.java	2011-05-22 07:40:42 +0000
+++ src/au/edu/unimelb/csse/mugle/server/model/GameVersionGetter.java	2011-05-22 18:34:23 +0000
@@ -155,6 +155,21 @@
     }
     
     /**
+     * Gets the GameVersion by its Primary Key - READ ONLY
+     * @param primaryKey
+     * @return the GameVersion - read only
+     * @throws GameVersionNotExists 
+     */
+    public static GameVersionData getGameVersion(Key primaryKey) throws GameVersionNotExists {
+        PersistenceManager pm = PMF.getManager();
+        try {
+            return getGameVersion(pm, primaryKey);
+        } finally {
+            pm.close();
+        }
+    }
+
+    /**
      * Gets the GameVersion associated with the given GameVersionData primary key.
      * @param pm The PersistenceManager
      * @param key The server-side key (ModelDataClass.getServerKey())
@@ -164,16 +179,11 @@
     public static GameVersionData getGameVersion(PersistenceManager pm, Key key)
             throws GameVersionNotExists {
         
-       try {
-           GameVersionData result = pm.getObjectById(GameVersionData.class, key);
-           if (result == null) {
-               throw new GameVersionNotExists(key.getId());
-           }
-           return result;
-       } finally {
-           pm.close();
+       GameVersionData result = pm.getObjectById(GameVersionData.class, key);
+       if (result == null) {
+           throw new GameVersionNotExists(key.getId());
        }
-       
+       return result;
     }
     
 }

=== added file 'src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameFileContentsNotExists.java'
--- src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameFileContentsNotExists.java	1970-01-01 00:00:00 +0000
+++ src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameFileContentsNotExists.java	2011-05-22 18:34:23 +0000
@@ -0,0 +1,15 @@
+package au.edu.unimelb.csse.mugle.shared.platform.exceptions;
+
+public class GameFileContentsNotExists extends MugleException {
+
+    private static final long serialVersionUID = 728229816363066583L;
+
+    public GameFileContentsNotExists() {
+        super();
+    }
+    
+    public GameFileContentsNotExists(Long primaryKey) {
+        super("The file contents with primary key " + primaryKey.toString() +
+              " does not exist.");
+    }
+}

=== modified file 'src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameFileNotExists.java'
--- src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameFileNotExists.java	2011-05-20 06:35:39 +0000
+++ src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameFileNotExists.java	2011-05-22 18:34:23 +0000
@@ -13,7 +13,12 @@
     }
     
     public GameFileNotExists(String gameName, String gameVersion, String path) {
-        super("The file " + path + " for the version " + gameVersion + 
-                        " of " + gameName + " does not Exist.");
+        super("The file " + path + " for version " + gameVersion + 
+                        " of " + gameName + " does not exist.");
+    }
+
+    public GameFileNotExists(String gameVersion, String path) {
+        super("The file " + path + " for version " + gameVersion +
+                        " of <unknown game> does not exist.");
     }
 }

=== modified file 'src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameNotExists.java'
--- src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameNotExists.java	2011-05-22 07:40:42 +0000
+++ src/au/edu/unimelb/csse/mugle/shared/platform/exceptions/GameNotExists.java	2011-05-22 18:34:23 +0000
@@ -38,6 +38,11 @@
         super("Game '" + gameName + "' does not exist.");
     }
     
+    public GameNotExists(String teamName, String gameName) {
+        super("Game '" + gameName + "' by team '" + teamName
+              + "' does not exist.");
+    }
+    
     public GameNotExists(Long id) {
         super("Game with ID '" + id + "' does not exist.");
     }

=== added file 'war/WEB-INF/lib/commons-fileupload-1.2.2.jar'
Binary files war/WEB-INF/lib/commons-fileupload-1.2.2.jar	1970-01-01 00:00:00 +0000 and war/WEB-INF/lib/commons-fileupload-1.2.2.jar	2011-05-22 18:34:23 +0000 differ
=== modified file 'war/WEB-INF/web.xml'
--- war/WEB-INF/web.xml	2011-05-08 06:05:33 +0000
+++ war/WEB-INF/web.xml	2011-05-22 18:34:23 +0000
@@ -132,6 +132,28 @@
     <servlet-name>UserGameProfileService</servlet-name>
     <url-pattern>/mugle/data-ugp</url-pattern>
   </servlet-mapping>
+
+  <!-- Game upload and hosting services -->
+
+  <servlet>
+    <servlet-name>UploadService</servlet-name>
+    <servlet-class>au.edu.unimelb.csse.mugle.server.UploadService</servlet-class>
+  </servlet>
+
+  <servlet-mapping>
+    <servlet-name>UploadService</servlet-name>
+    <url-pattern>/mugle/upload</url-pattern>
+  </servlet-mapping>
+
+  <servlet>
+    <servlet-name>GameFileServer</servlet-name>
+    <servlet-class>au.edu.unimelb.csse.mugle.server.GameFileServer</servlet-class>
+  </servlet>
+
+  <servlet-mapping>
+    <servlet-name>GameFileServer</servlet-name>
+    <url-pattern>/+games/*</url-pattern>
+  </servlet-mapping>
   
   <!-- API servlet mappings -->