HTTP Authentication Debate: Revolution?

Platform wars aside, serious disagreements among highly-accomplished software engineers rarely involve much drama. As in the sciences generally, the professional and career incentives normally tend to favor consensus and steady progress at the margins. But once in a while, things do come to a head.

This might be one of those times, at least among the small minority of technical people who are intensely concerned with the underlying protocol of the World Wide Web.

At a recent Internet Engineering Task Force (IETF) session, a major debate broke out regarding what to do about HTTP. At issue is whether to make much needed but “minimal corrections” to the existing HTTP 1.1 specification (RFC 2616 ) — mostly fixing known but minor errors, as in this draft — or instead to undertake a major revision. The latter would involve not only updating RFC 2616, but merging it with a thoroughly revised version of a completely different document: RFC 2617. This is the spec for a key HTTP security mechanism — one with very few fans: HTTP authentication.

In a nutshell, the group arguing for major changes says that HTTP authentication is broken, and they want to take HTTP itself back into the shop in order to fix it. For this group, that means coming up with a form of HTTP auth that works well enough so that developers will actually start using it. Because the essential problem with HTTP auth — the thing that condemns it and makes some call for an HTTP revolution to replace it — is that hardly anyone does, in fact, use it.

If it is surprising that a key part of the HTTP protocol’s security scheme has been effectively abandoned, it shouldn’t be: How do you do authentication on your sites and online apps? Chances are, you use forms, session cookies, SSL encryption, and hashed credentials in your provider of choice. Chances are, you don’t rely on the two alternatives provided by RFC 2617, namely, basic auth and digest auth. But why not? How strong is the case for revolutionary change in how HTTP does authentication?

The bill of particulars against HTTP auth is a long once, with several levels. Some of the issues turn out to be not that crucial. Others have more staying power. Let’s take a closer look.

At the simplest level, both HTTP auth schemes natively lack confidentiality for the login request. In basic auth, everything including the password goes over the wire in clear text (the password goes in Base64 encoding, which is trivial to decode). Used without SSL, it is an invitation to both spoofing and replay attacks.

Digest auth improves on this somewhat, by not having the client transmit the secret (the password) at all, but instead having it send a hash of the secret made with a seed value (a nonce) supplied by the server. Since the secret isn’t transmitted, it can’t be sniffed and easily spoofed. Since the server-supplied nonce can be built from the specific resource (URI), client (IP), time period (timestamp plus time limit), and other restrictions, it greatly complicates replay attacks. Basically, the constantly changing nonces help deter replay attacks because the hash of the client’s password will be different for each logon.

All well and good, but there are still major problems. Even disgest auth leaves the entire request except for the credentials in the clear — used without SSL, it still leaves an opening for more sophisticated replay attacks. Worse, it is (again without SSL) just as vulnerable as basic auth to Man in the Middle attacks (e.g., malicious proxies that break the authentication chain, sometimes simply by using the HTTP auth negotiation mechanism to ‘negotiate down’ to the least-secure auth method available).

Well, you might be thinking at this point, this would all be pretty devastating, but doesn’t most of it come down to saying the HTTP auth requires the use of SSL to encrypt the channel? And once that encryption is added, don’t these problems mostly go away? And don’t we generally use SSL for our forms-and-cookies based authentication anyway (at least if we know what we’re doing)? So why is HTTP auth still so little used?

And that brings us to the real problems, which have to do with implementation issues and user experience. First, digest auth (the superior and recommended of the two alternatives) is notoriously under-supported on the browser side, and to some degree dependent on browsers settings, even when fully supported. Second, it also requires special mechanisms server-side (e.g., for Windows, an Active Directory domain). Third, because the passwords, though not sent over the wire, must be available as a hash input server-side, they must be persisted there, at best in reversibly-encrypted form, thus creating a vulnerability that smart developers avoid in forms-and-cookies based schemes by storing hashes only.

But even if all these issues could be overcome, there would still be one glaring problem that would deter widespread adoption of HTTP auth, at least for publicly-accessible sites and applications, namely, the dreaded challenge/response dialog. HTTP auth requires the clinet to launch this dialog to get the user’s credentials, and it leaves the look-and-feel and behavior of that dialog up to the client — in practice, the OS. The resulting sharp break (more like jolt) in the user’s experience of the site is usually too much for the site owner or stakeholders to bear.

And there you have it — the case against HTTP auth as it stands. So what do you think? Are you ready to unite under the banner of the HTTP Auth revolutionaries? If so, get yourself to the IETF/W3C barricades and sound the call, “You have nothing to lose but your session cookies!”

(For those interested in pursuing this further, Paul James has come up with an XHR-based technique for using HTTP auth in the background, with HTML forms as the user-visible component. Check it out here.)