True OAuth Confessions, or Why My Hand-Rolled Calls All Blew Chunks .:. kentbrewster.com

I'm settling in at Netflix and starting to work my way through the developer docs. One of the main concerns we're seeing from outside developers is getting that first OAuth call to work. Here, in an effort to reassure my fellow noobs that they're not crazy, is my tale of OAuth woe:

When I first encountered OAuth I bounced off the spec and a couple of libraries, which seem to be documented in a language created specifically to make guys like me give up in disgust.

Backup plan: I cracked into a couple of URL returns with Firebug, thought I understood what was going on, and plunged blindly in. (Famous last words: "It's just a string, right? How hard could it possibly be?")

My real mistake: not just blindly trusting the libraries to do the right thing. Seriously. Unless you've got the same thing wrong with you that I have with me and you absolutely cannot stand the idea of using something you don't fully understand, stop here and just go plug in a library.

So. Here begins my top ten list of horrible mistakes made while trying to reverse-engineer OAuth:

  • My Timestamp was Stale
    Unless it's only a few minutes old, the URL somebody else generated isn't going to work. And forget about trying to copy and paste the one they show in the docs. If you're working in JavaScript, your client's clock needs to be within ten minutes of your API's clock. (WinXP under Parallels seems to lose track of time every once in a while; cue the sound of Nelson Muntz from The Simpsons: "Ha-hah!")
  • The Parameters in my Signature Base String were Out of Alphabetical Order
    Note to self: oauth_signature_method comes BEFORE oauth_timestamp, and AFTER oauth_nonce. Parameters like foo and baz and qux might show up before or afterwards. Or during, depending on how psychotic the API you're trying to hit turns out to be.
  • I Forgot a Question-Mark between my Request and my OAuth Payload
    Here's where you do NOT want an ampersand. You'll go blind trying to spot it.
  • I Didn't URL-Encode my Signature
    Just because it came back in base-64 from my HMAC-SHA1 function didn't make it safe to fire off; there are plus-signs and equals-signs and other unsavory characters in many base-64 strings, and they all needed to be encoded.
  • I Didn't URL-Encode All Illegal Characters
    Right, see, I was initially doing this with JavaScript, and uriEncodeComponent turned out not to fix exclamation points, asterisks, single-quotes, or parentheses.
  • I URL-Encoded the Ampersands Separating my Method, URL, and Request Parameters
    The server needs those raw ampersands, so it can tell where to split the string you sent.
  • I Didn't Append an Ampersand to my Consumer Secret to Make my Signature Key
    Another picky ampersand-related detail: when signing a request without a token secret I STILL needed to add an ampersand to the end of my consumer secret.
  • I Used the Same Nonce, Over and Over and Over Again
    Turns out there is a special circle of Error 401 Hell reserved for people who re-use their nonces; this makes sense, if you think about the sort of replay attacks the spec is designed to prevent. Naturally you would need to READ the spec before thinking about this.
  • I Generated a Random Nonce, but (you guessed it) Failed to URL-Encode It
    No need to be fancy with this; just pick a random 16-digit number. Don't (for instance) use every possible printable character.
  • And, Finally: I URL-Encoded my Signature Key
    Yeah, I really did do this. It took days to unscrew. Not recommended.

Anyone else have something to confess? Let's hear it; it will be good for your soul, and will help our fellow pilgrims not to fall down the same holes we did.

Comments from before Disqus:

Paul Tarjan .:. 2010-07-20 20:19:11
I made you famous and quoted you in a Facebook doc:

http://developers.facebook.com/docs/authentication/canvas
Sam Pullara .:. 2010-04-04 23:22:19
The biggest fun that I had was with Rhino (the Mozilla Java-based Javascript implementation) that uses lowest case hex for encoding rather than uppercase hex as the specification requires.
Erich .:. 2010-03-15 20:44:02
My confession. My first problem with the API was dealing with spaces in search terms. That's when I noticed double url encoding was needed. I promptly forgot this and was failing to double url encode arguments (non-oauth arguments) like movie id strings, etc. Now it works. And I wrote all my own oauth code just to learn it better.
Erich .:. 2010-03-15 04:34:16
Kent-
I have everything working in oauth except POST. I retrieve etags so I know that all my signing is good for subscriber-based functions. But when I try POST, it breaks. The questions are:
what goes in the body?
what stays in the URL (as part of the query string, if anything)
what does netflix work with regarding sending of the auth parameters? I know there are 3 ways.

any web sites for testing POST commands?
Kenan Banks .:. 2009-08-17 00:21:20
I hate the same thing wrong with me that you have wrong with you. I just went through this checklist and things still aren't working. 3:21am here. Wish me luck. Or sleep.
Kent Brewster .:. 2009-04-29 09:44:35
LG: if you're messing with the Netflix API, try out my new Netflix API Explorer. It will sign catalog calls and give you back URLs; compare these to what your libraries are generating and you might find the difference.
LG .:. 2009-04-29 08:05:17
Kent, Thanks for the pointers very useful, feeling the oAuth sig pain at the mo. myself. Feeling like a life hurdle hair's getting greyer by the hour...
kellan .:. 2009-04-28 10:35:02
Kent, this is a great list/resource.

Allen, I still maintain the difficulty with signatures is the complexity of the docs, not the ideas. We see a fair number of folks each second successfully signing FlickrAuth calls.
jrconlin .:. 2009-04-27 18:25:22
Well, it was a bit late in coming out, but I did try and build a javascript based OAuth Test page at http://developer.netflix.com/resources/OAuthTest which allows you to play with OAuth a bit better than just dorking around blindly.

OAuth (like a lot of new stuff) is complicated until you know how it works, then it's easy. Then again, a lightbulb is just a hot wire in a jar and look how long that took.
Allen Tom .:. 2009-04-27 16:58:19
Would it be easier if we dropped the signature requirement, provided that the Access Token doesn't live that long? It seems that most of the OAuth grief is due to the extreme difficulty in generating signatures, and at least for Yahoo's implementation of OAuth, we would be OK with dropping most of the sigs if the Access Tokens expire within an hour or so.

This would essentially make OAuth equivalent to Yahoo's BBAuth