Migrate users from Google App Engine to Google OpenID - 【StackMirror】|python|django|google-app-engine|openid

I migrated away from Google App Engine several months ago. But I am still relying on it for authentication, because my users are identified by their user_id attribute on GAE.

For this purpose my (now external) applications redirect the user to a Google App Engine application using a encrypted, signed and timestamped login request. The GAE application then performs the login using GAE's "Users" service. After successfully being logged-in on GAE, the user is again redirected using a encrypted, signed and timestamped response to my external application. The rudimentary implementation can be found here and here. As you can see, this is very basic and relies on heavy crypto that leads to bad performance.

My external applications, in this case Django applications, are storing the user_id inside the password field of the user table. Besides the user_id, I only get the email address from GAE to store username and email in Django.

Now I would like to remove the dependency on the GAE service. The first approach which comes to mind would probably be to send an email to each user requesting him to set a new password and then perform my own authentication using Django.

I would prefer a solution which relies on Google's OpenID service so that there is actually no difference for the user. This is also preferred, because I need to send the user to Google anyway to get AuthSub tokens for the Google Calendar API.

The problem is that I couldn't find a way to get the GAE user_id attribute of a given Google Account without using GAE. OpenID and all the other authentication protocols use different identifiers.

So now the question is: Does Google provide any API I could use for this purpose which I haven't seen yet? Are there any other possible solutions or ideas on how to migrate the user accounts?

Thanks in advance!

2012-04-04 07:32
by mback2k

The best way to do this is to show users a 'migration' interstital, which redirects them to the Google OpenID provider and prompts them to sign in there. Once they're signed in at both locations, you can match the two accounts, and let them log in over OpenID in future.

2012-04-05 05:58
by Nick Johnson

Thanks for the answer, but as I now pointed out in the other comments: I would really like to avoid a transition phase, migration period or anything like that. And I also want to make sure that non-one is left behind. (I know that your solution is fine, but others might not.) One thing I should probably add is that, even though I store the userid in the password field of my Django authuser table, the user_id is not encrypted or hashed. If an API to convert them to OpenID identifiers or anything like that would exist, I could migrate easily - mback2k 2012-04-05 06:08

@mback2k What you're looking for doesn't exist. The user ID returned by the App Engine Users API is an internal identifier; there's no external correspondence to it, or any way to convert it to an external identifier - Nick Johnson 2012-04-05 06:10

Ok, I see. Thanks for the answer. Could this be something the GAE team might be looking into in the future? As an element of the open data import/export and migration efforts - mback2k 2012-04-05 06:14

@mback2k Unfortunately, I don't think it's something we can offer, because it would compromise user privacy - it would associate a user's openid account (which doesn't automatically provide any information about them unless they grant it) with their Google account, without them having to provide explicit permission to do so - Nick Johnson 2012-04-05 06:18

Are you sure about that? Wouldn't it be the other way around. I mean, I already have the identifier to their Google Account that theoretically gives me more information and now I would like to go back to a more general OpenID identifier with less information - mback2k 2012-04-05 06:27

@mback2k Suppose Joe Blackhat compromises your App Engine app and downloads its datastore. With this API, he now knows the email address of anyone using his (unrelated) app who signs in with a Google OpenID account. Implicitly, he could also use the API to compare OpenIDs between apps, even though Google's OpenID provider deliberately provides different IDs for different apps - Nick Johnson 2012-04-05 06:29

Forgive me, but either I still don't see it or I think the solution is easy. First of all it would be a one-way lookup service: GAE user_id -> new Google OpenID identifier generated for my app (eg. API key). Do you mean that the problem is that I already got the email address without permission and that it could be used to match email addresses to OpenIDs? Please enlighten me. ;- - mback2k 2012-04-05 06:35

Google OpenID identifiers are one per app, to the best of my knowledge - you can't generate new ones. If you can map from a user ID to an app's OpenID ID, then you as an App Engine app owner can connect a user's identity on multiple sites - Nick Johnson 2012-04-05 06:37

Two things to consider: 1. There would be no change, because the user ID already allows me to connect a user's identity on multiple sites since it is globally unique for all GAE apps. 2. Yes, Google OpenID identifiers are one per app and I see that as a solution to the problems you describe, not a problem source. Remember, I am coming from the global unique identifier to the per-app unique identifier. (Reverse lookup shouldn't be possible unless you are able to make all Google Accounts use one of your GAE apps and then bruteforce using the API. But still, the new tokens would be per-app. - mback2k 2012-04-05 06:40

If such an API existed, you, as an app engine app owner, could call convert_userid(user, 'domaina.com') and convert_userid(user, 'domainb.com'). Now you know the OpenID of the user on two different sites, completely bypassing the point of Google's OpenID issuing different URLs per site - Nick Johnson 2012-04-05 06:45

Ah, thanks, that made me understand the real problem. So maybe there needs to be a limitation and some kind of verification on which domains you are allowed to get OpenID tokens for. Some kind of domain protection/verification, just like it's used for OAuth 1.0 Sites here - mback2k 2012-04-05 06:51

@mback2k But now you're talking about a major endeavor for a fairly limited use-case - Nick Johnson 2012-04-05 08:49

Yes, that's true. But do me a favor and put it on your list somewhere. Maybe it can be done at some point and maybe existing technologies can be used to reduce the overall implementation workload - mback2k 2012-04-05 08:57

I got a new idea: What do you think about adding the GAE user_id identifier to the Google OAuth 2.0 Profile API? That way the old accounts can be matched to the different Google Plus+/Profile ID. https://www.googleapis.com/oauth2/v1/userinf - mback2k 2012-06-10 13:26

AFAIK, the only common identifier between Google Accounts and Google OpenID is the email.

Get email when user logs into Google Account via your current gae setup. Use User.email(). Save this email along with the user data.
When you have emails of all (most) users, switch to Google OpenID. When user logs in, get the email address and find this user in the database.

2012-04-04 13:37
by Peter Knego

Thanks for the response. The problem with this is that a user could have changed his email address in the meantime, for example from a non-gmail address to gmail, or to another non-gmail one. I experienced this before and had to manually provide a way for users to contact me and re-link their old account and associated data. The user_id is a static identifier which never changes and I would really like to make sure that no one is left behind - mback2k 2012-04-04 14:34

Why don't you try a hybrid approach:

Switch to OpenId
If your application already knows the userId, you are done
If not ask the user, if he has an account to migrate
If yes, log him in with the old mechansim and ttransfer the acount
If not create a new account

2012-04-25 20:30
by Ruediger Jungbeck

Google has a unique identifier that's returned as a parameter with a successful OpenID authentication request - *openid.claimed_id* . If you switch to using OpenID you could essentially exchange the user_id for this parameter the first time a user logs in using the new method without the user noticing anything different about their login experience.

Documentation for the authentication process is outlined here. I'd recommend using the hybrid OpenID+OAuth approach so that you can associate your request token with a given id, then, upon return, verify that the openid.claimed_id matches your original request token.

2012-04-04 19:07
by Kevin P

Thanks. Yes, but how would I match existing users to their accounts. The point is that the email address or nickname is not unique and static. Nearly 15% of my user base are using non-gmail/googlemail.com addresses which could change at any point in time. I am currently using the user_id as the single factor for user identification, but keep email and nickname for information and contact purposes - mback2k 2012-04-04 19:42

Two approaches possible. One, you simply associate their account on your system with whatever google account they log into the first time. Two (more secure) during the transition period you authenticate users first using the current method and then again using the OpenID method - Kevin P 2012-04-04 20:26

One other thing to add/clarify - you should generate your own unique identifier for users before you start the transition and not rely on the Google identifiers for anything other than validating OpenID-based authentication going forward - Kevin P 2012-04-04 20:36

One doesn't work reliable, because of the email identifier issues. People changing their email address in between means that the connection to the data on my service is lost. Two is probably the only option right now, even though I would prefer a solution without an actual transition period. Yes, sure, once the user accounts are real Django accounts they will be identified by username and/or email address. OpenID would just exist for convenience. Thanks again, but I am still interested in other ideas - mback2k 2012-04-05 06:03