KeyValue service errors when first creating UserGameProfile

I have a feeling i know what the problem is here.
Can you write a test game where the only functionality is to put a
KeyValuePair?

If what i suspect is correct;

KeyValuePairs and UserGameProfiles are not in the same entity group - so the
first time a game is started up, what happens when a call to the
KeyValuePair service is made is;

PersistenceManager pm = PMF.getManager();
//lookup UGP, not found so create a new one
//KVP not found so create a new one
pm.close()

Since KeyValuePairs and UserGameProfiles are in different entity groups,
this cant be done as the Google App Engine error indicates.

There are two potential ways around this;

1) Try closing the PersistenceManager between the two creation calls.

2) If this doesnt work it would be best to make the relationship between
UGPs and KeyValuePairs owned (being in an owned relationship forces these
two datatypes to be in the same entity group)
The limitation of this is Owned relationships can only handle very simple
database models, which is why it didnt work for ours initially and we had to
rewrite the backend to used unowned relationships.

On Thu, May 26, 2011 at 10:37 PM, Matt Giuca <email address hidden>wrote:

> Public bug reported:
>
> I'm not quite sure what's happening now. This doesn't happen in my API
> test, but in one of the student games. The game makes many requests to
> the KeyValue service while it is starting up (perhaps that's the
> difference -- APITest only makes requests after some length of time).
>
> Every one of the requests fails with a server error, but this only
> happens the first time a user plays that particular game. On all
> subsequent plays, it is fine.
>
> The message on the server is:
> Caused by: javax.jdo.JDOFatalUserException: Illegal argument
> NestedThrowables: java.lang.IllegalArgumentException: can't operate on
> multiple entity groups in a single transaction. found both Element { type:
> "UserGameProfileData" id: 389 } and Element { type: "GameData" id: 3 }
> at
> au.edu.unimelb.csse.mugle.server.model.KeyValuePairGetter.getKeyValuePair(KeyValuePairGetter.java:51)
> at
> au.edu.unimelb.csse.mugle.server.api.KeyValueServiceImpl.get(KeyValueServiceImpl.java:58)
>
> ** Affects: mugle
> Importance: Critical
> Assignee: Matt Giuca (mgiuca)
> Status: In Progress
>
>
> ** Tags: datastore
>
> --
> You received this bug notification because you are subscribed to MUGLE.
> https://bugs.launchpad.net/bugs/788592
>
> Title:
> KeyValue service errors when first creating UserGameProfile
>
> Status in Melbourne University Game-based Learning Environment:
> In Progress
>
> Bug description:
> I'm not quite sure what's happening now. This doesn't happen in my API
> test, but in one of the student games. The game makes many requests to
> the KeyValue service while it is starting up (perhaps that's the
> difference -- APITest only makes requests after some length of time).
>
> Every one of the requests fails with a server error, but this only
> happens the first time a user plays that particular game. On all
> subsequent plays, it is fine.
>
> The message on the server is:
> Caused by: javax.jdo.JDOFatalUserException: Illegal argum...

Ok just read the other bug reports - and that seems to indicate what i said
is definately the case.

Looks like we'll need to go and track down wherever a UserGameProfile,
UserAchievement or KeyValuePair is made and make sure that the
persistencemanager isnt kept open between writes.
These three should be the only cases where two objects are created at once.

On Thu, May 26, 2011 at 10:56 PM, Scott Ritchie <email address hidden>wrote:

> I have a feeling i know what the problem is here.
> Can you write a test game where the only functionality is to put a
> KeyValuePair?
>
> If what i suspect is correct;
>
> KeyValuePairs and UserGameProfiles are not in the same entity group - so
> the first time a game is started up, what happens when a call to the
> KeyValuePair service is made is;
>
> PersistenceManager pm = PMF.getManager();
> //lookup UGP, not found so create a new one
> //KVP not found so create a new one
> pm.close()
>
> Since KeyValuePairs and UserGameProfiles are in different entity groups,
> this cant be done as the Google App Engine error indicates.
>
> There are two potential ways around this;
>
> 1) Try closing the PersistenceManager between the two creation calls.
>
> 2) If this doesnt work it would be best to make the relationship between
> UGPs and KeyValuePairs owned (being in an owned relationship forces these
> two datatypes to be in the same entity group)
> The limitation of this is Owned relationships can only handle very simple
> database models, which is why it didnt work for ours initially and we had to
> rewrite the backend to used unowned relationships.
>
>
> On Thu, May 26, 2011 at 10:37 PM, Matt Giuca <email address hidden>wrote:
>
>> Public bug reported:
>>
>> I'm not quite sure what's happening now. This doesn't happen in my API
>> test, but in one of the student games. The game makes many requests to
>> the KeyValue service while it is starting up (perhaps that's the
>> difference -- APITest only makes requests after some length of time).
>>
>> Every one of the requests fails with a server error, but this only
>> happens the first time a user plays that particular game. On all
>> subsequent plays, it is fine.
>>
>> The message on the server is:
>> Caused by: javax.jdo.JDOFatalUserException: Illegal argument
>> NestedThrowables: java.lang.IllegalArgumentException: can't operate on
>> multiple entity groups in a single transaction. found both Element { type:
>> "UserGameProfileData" id: 389 } and Element { type: "GameData" id: 3 }
>> at
>> au.edu.unimelb.csse.mugle.server.model.KeyValuePairGetter.getKeyValuePair(KeyValuePairGetter.java:51)
>> at
>> au.edu.unimelb.csse.mugle.server.api.KeyValueServiceImpl.get(KeyValueServiceImpl.java:58)
>>
>> ** Affects: mugle
>> Importance: Critical
>> Assignee: Matt Giuca (mgiuca)
>> Status: In Progress
>>
>>
>> ** Tags: datastore
>>
>> --
>> You received this bug notification because you are subscribed to MUGLE.
>> https://bugs.launchpad.net/bugs/788592
>>
>> Title:
>> KeyValue service errors when first creating UserGameProfile
>>
>> Status in Melbourne University Game-based Learning Environment:
>> In Progress
>>
>> Bug description:
>> I'm not quite sur...

So this can be traced down to UserGameProfileGetter.getCurrentUserGameProfile(PersistenceManager, String), which does the following, if and only if the UGP does not yet exist:
1. Get a UserData from the datastore (with the same persistence manager),
2. Get a GameData from the datastore (with the same persistence manager),
3. Create a new UserGameProfileData,
4. Write the UserGameProfileData to the datastore (with the same persistence manager),
5. Update the UserData to refer to the UserGameProfileData,
6. Update the GameData to refer to the UserGameProfileData.

Then, in KeyValuePairGetter:51, it calls pm.close() which brings everything crashing down.

It seems like the problem is that the *same* persistence manager is being used to write to both the GameData *and* the UserGameProfileData. The problem goes away if you change step #1 and #2 to each create their own PM (trivial -- just remove the PM argument to the call).

I'm quite surprised about this. I knew that a *transaction* could only apply to a single entity group, but I had no idea that a persistence manager could only apply to one entity group for its lifetime. I am surprised that this hasn't come up elsewhere. We should be on the look out for it.

Yeah it's actually quite suprising that this is the first time it has come
up - i didnt get this error when writing the populated datastore button, i
was quite happily creating multiple different entities using the same
PersistenceManager.

On Thu, May 26, 2011 at 11:11 PM, Matt Giuca <email address hidden>wrote:

> So this can be traced down to
> UserGameProfileGetter.getCurrentUserGameProfile(PersistenceManager, String),
> which does the following, if and only if the UGP does not yet exist:
> 1. Get a UserData from the datastore (with the same persistence manager),
> 2. Get a GameData from the datastore (with the same persistence manager),
> 3. Create a new UserGameProfileData,
> 4. Write the UserGameProfileData to the datastore (with the same
> persistence manager),
> 5. Update the UserData to refer to the UserGameProfileData,
> 6. Update the GameData to refer to the UserGameProfileData.
>
> Then, in KeyValuePairGetter:51, it calls pm.close() which brings
> everything crashing down.
>
> It seems like the problem is that the *same* persistence manager is
> being used to write to both the GameData *and* the UserGameProfileData.
> The problem goes away if you change step #1 and #2 to each create their
> own PM (trivial -- just remove the PM argument to the call).
>
> I'm quite surprised about this. I knew that a *transaction* could only
> apply to a single entity group, but I had no idea that a persistence
> manager could only apply to one entity group for its lifetime. I am
> surprised that this hasn't come up elsewhere. We should be on the look
> out for it.
>
> --
> You received this bug notification because you are subscribed to MUGLE.
> https://bugs.launchpad.net/bugs/788592
>
> Title:
> KeyValue service errors when first creating UserGameProfile
>
> Status in Melbourne University Game-based Learning Environment:
> In Progress
>
> Bug description:
> I'm not quite sure what's happening now. This doesn't happen in my API
> test, but in one of the student games. The game makes many requests to
> the KeyValue service while it is starting up (perhaps that's the
> difference -- APITest only makes requests after some length of time).
>
> Every one of the requests fails with a server error, but this only
> happens the first time a user plays that particular game. On all
> subsequent plays, it is fine.
>
> The message on the server is:
> Caused by: javax.jdo.JDOFatalUserException: Illegal argument
> NestedThrowables: java.lang.IllegalArgumentException: can't operate on
> multiple entity groups in a single transaction. found both Element { type:
> "UserGameProfileData" id: 389 } and Element { type: "GameData" id: 3 }
> at
> au.edu.unimelb.csse.mugle.server.model.KeyValuePairGetter.getKeyValuePair(KeyValuePairGetter.java:51)
> at
> au.edu.unimelb.csse.mugle.server.api.KeyValueServiceImpl.get(KeyValueServiceImpl.java:58)
>

Fixed in trunk r463.

Sorry Scott, I only just saw your replies now. They would have been useful. Oh well -- so I guess what you were preaching was right all along ;) Make separate PersistenceManagers where possible. You can't in general just chain one persistence manager through a whole bunch of calls.

But note that you don't need to close the PM between calls. You can have multiple PMs on the go at once, just can't use the same one for multiple entity groups.