nin page 6 - Dave's Blog

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Getting into the more subtle levels of URI percent-encoding ignorance, folks try to apply their knowledge of percent-encoding to URIs as a whole producing the concepts escaped URIs and unescaped URIs. However there are no such things - URIs themselves aren't percent-encoded or decoded but rather contain characters that are percent-encoded or decoded. Applying percent-encoding or decoding to a URI as a whole produces a new and non-equivalent URI.

Instead of lingering on the incorrect concepts we'll just cover the correct ones: there's raw unencoded data, non-normal form URIs and normal form URIs. For example:

1. http://example.com/%74%68%65%3F%70%61%74%68?query
2. http://example.com/the%3Fpath?query
3. "http", "example.com", "the?path", "query"

In the above (A) is not an 'encoded URI' but rather a non-normal form URI. The characters of 'the' and 'path' are percent-encoded but as unreserved characters specific in the RFC should not be encoded. In the normal form of the URI (B) the characters are decoded. But (B) is not a 'decoded URI' -- it still has an encoded '?' in it because that's a reserved character which by the RFC holds different meaning when appearing decoded versus encoded. Specifically in this case, it appears encoded which means it is data -- a literal '?' that appears as part of the path segment. This is as opposed to the decoded '?' that appears in the URI which is not part of the path but rather the delimiter to the query.

Usually when developers talk about decoding the URI what they really want is the raw data from the URI. The raw decoded data is (C) above. The only thing to note beyond what's covered already is that to obtain the decoded data one must parse the URI before percent decoding all percent-encoded octets.

Of course the exception here is when a URI is the raw data. In this case you must percent-encode the URI to have it appear in another URI. More on percent-encoding while constructing URIs later.

Shortly after joining the Internet Explorer team I got a bug from a PM on a popular Microsoft web server product that I'll leave unnamed (from now on UWS). The bug said that IE was handling empty path segments incorrectly by not removing them before resolving dotted path segments. For example UWS would do the following:

A.1. http://example.com/a/b//../
A.2. http://example.com/a/b/../
A.3. http://example.com/a/

In step 1 they are given a URI with dotted path segment and an empty path segment. In step 2 they remove the empty path segment, and in step 3 they resolve the dotted path segment. Whereas, given the same initial URI, IE would do the following:

B.1. http://example.com/a/b//../
B.2. http://example.com/a/b/

IE simply resolves the dotted path segment against the empty path segment and removes them both. So, how did I resolve this bug? As "By Design" of course!

The URI RFC allows path segments of zero length and does not assign them any special meaning. So generic user agents that intend to work on the web must not treat an empty path segment any different from a path segment with some text in it. In the case above IE is doing the correct thing.

That's the case for generic user agents, however servers may decide that a URI with an empty path segment returns the same resource as a the same URI without that empty path segment. Essentially they can decide to ignore empty path segments. Both IIS and Apache work this way and thus return the same resource for the following URIs:

http://exmaple.com/foo//bar///baz
http://example.com/foo/bar/baz

The issue for UWS is that it removes empty path segments before resolving dotted path segments. It must follow normal URI procedure before applying its own additional rules for empty path segments. Not doing that means they end up violating URI equivalency rules: URIs (A.1) and (B.2) are equivalent but UWS will not return the same resource for them.

A bug came up the other day involving markup containing <input type="image" src="http://example.com/.... I knew that "image" was a valid input type but it wasn't until that moment that I realized I didn't know what it did. Looking it up I found that it displays the specified image and when the user clicks on the image, the form is submitted with an additional two name value pairs: the x and y positions of the point at which the user clicked the image.

Take for example the following HTML:

<form action="http://example.com/">
<input type="image" name="foo" src="http://deletethis.net/dave/images/davebefore.jpg">
</form>

If the user clicks on the image, the browser will submit the form with a URI like the following:http://example.com/?foo.x=145&foo.y=124.

This seemed like an incredibly specific feature to be built directly into the language when this could instead be done with javascript. I looked a bit further and saw that its been in HTML since at least HTML2, which of course makes much more sense. Javascript barely existed at that point and sending off the user's click location in a form may have been the only way to do something interesting with that action.