val page 6 - Dave's Blog

By the URI RFC there is only one way to represent a particular IPv4 address in the host of a URI. This is the standard dotted decimal notation of four bytes in decimal with no leading zeroes delimited by periods. And no leading zeros are allowed which means there's only one textual representation of a particular IPv4 address.

However as discussed in the URI RFC, there are other forms of IPv4 addresses that although not officially allowed are generally accepted. Many implementations used inet_aton to parse the address from the URI which accepts more than just dotted decimal. Instead of dotted decimal, each dot delimited part can be in decimal, octal (if preceded by a '0') or hex (if preceded by '0x' or '0X'). And that's each section individually - they don't have to match. And there need not be 4 parts: there can be between 1 and 4 (inclusive). In case of less than 4, the last part in the string represents all of the left over bytes, not just one.

For example the following are all equivalent:

192.168.1.1

Standard dotted decimal form

0300.0250.01.01

Octal

0xC0.0XA8.0x1.0X1

Hex

192.168.257

Fewer parts

0300.0XA8.257

All of the above

The bread and butter of URI related security issues is when one part of the system disagrees with another about the interpretation of the URI. So this non-standard, non-normal form syntax has been been a great source of security issues in the past. Its mostly well known now (CreateUri normalizes these non-normal forms to dotted decimal), but occasionally a good tool for bypassing naive URI blocking systems.

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Getting into the more subtle levels of URI percent-encoding ignorance, folks try to apply their knowledge of percent-encoding to URIs as a whole producing the concepts escaped URIs and unescaped URIs. However there are no such things - URIs themselves aren't percent-encoded or decoded but rather contain characters that are percent-encoded or decoded. Applying percent-encoding or decoding to a URI as a whole produces a new and non-equivalent URI.

Instead of lingering on the incorrect concepts we'll just cover the correct ones: there's raw unencoded data, non-normal form URIs and normal form URIs. For example:

1. http://example.com/%74%68%65%3F%70%61%74%68?query
2. http://example.com/the%3Fpath?query
3. "http", "example.com", "the?path", "query"

In the above (A) is not an 'encoded URI' but rather a non-normal form URI. The characters of 'the' and 'path' are percent-encoded but as unreserved characters specific in the RFC should not be encoded. In the normal form of the URI (B) the characters are decoded. But (B) is not a 'decoded URI' -- it still has an encoded '?' in it because that's a reserved character which by the RFC holds different meaning when appearing decoded versus encoded. Specifically in this case, it appears encoded which means it is data -- a literal '?' that appears as part of the path segment. This is as opposed to the decoded '?' that appears in the URI which is not part of the path but rather the delimiter to the query.

Usually when developers talk about decoding the URI what they really want is the raw data from the URI. The raw decoded data is (C) above. The only thing to note beyond what's covered already is that to obtain the decoded data one must parse the URI before percent decoding all percent-encoded octets.

Of course the exception here is when a URI is the raw data. In this case you must percent-encode the URI to have it appear in another URI. More on percent-encoding while constructing URIs later.

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Worse than the lame blog comments hating on percent-encoding is the shipping code which can do actual damage. In one very large project I won't name, I've fixed code that decodes all percent-encoded octets in a URI in order to get rid of pesky percents before calling ShellExecute. An unnamed developer with similar intent but clearly much craftier did the same thing in a loop until the string's length stopped changing. As it turns out percent-encoding serves a purpose and can't just be removed arbitrarily.

Percent-encoding exists so that one can represent data in a URI that would otherwise not be allowed or would be interpretted as a delimiter instead of data. For example, the space character (U+0020) is not allowed in a URI and so must be percent-encoded in order to appear in a URI:

1. http://example.com/the%20path/
2. http://example.com/the path/

In the above the first is a valid URI while the second is not valid since a space appears directly in the URI. Depending on the context and the code through which the wannabe URI is run one may get unexpected failure.

For an additional example, the question mark delimits the path from the query. If one wanted the question mark to appear as part of the path rather than delimit the path from the query, it must be percent-encoded:

1. http://example.com/foo%3Fbar
2. http://example.com/foo?bar

In the second, the question mark appears plainly and so delimits the path "/foo" from the query "bar". And in the first, the querstion mark is percent-encoded and so the path is "/foo%3Fbar".

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping). The basest ignorance is with respect to the mere existence of percent-encoding. Percents in URIs are special: they always represent the start of a percent-encoded octet. That is to say, a percent is always followed by two hex digits that represents a value between 0 and 255 and doesn't show up in a URI otherwise.

The IPv6 textual syntax for scoped addresses uses the '%' to delimit the zone ID from the rest of the address. When it came time to define how to represent scoped IPv6 addresses in URIs there were two camps: Folks who wanted to use the IPv6 format as is in the URI, and those who wanted to encode or replace the '%' with a different character. The resulting thread was more lively than what shows up on the IETF URI discussion mailing list. Ultimately we went with a percent-encoded '%' which means the percent maintains its special status and singular purpose.

Cool and (relatively) new methods on the JavaScript Array object are here in the most recent versions of your favorite browser! More about them on ECMAScript5, MSDN, the IE blog, or Mozilla's documentation. Here's the list that's got me excited:

some & every

Does your callback function return true for any (some) or all (every) of the array's elements?

filter

Filters out elements for which your callback function returns false (in a new copy of the Array).

map

Each element is replaced with the result of it run through your callback function (in a new copy of the Array).

reduce & reduceRight

Your callback is called on each element in the array in sequence (from start to finish in reduce and from finish to start in reduceRight) with the result of the previous callback call passed to the next. Reduce your array to a single value aggregated in any manner you like via your callback function.

forEach

Simply calls your callback passing in each element of your array in turn. I have vague performance concerns as compared to using a normal for loop.

indexOf & lastIndexOf

Finds the first or last (respectively) element in the array that matches the provided value via strict equality operator and returns the index of that element or -1 if there is no such element. Surprisingly, no custom comparison callback method mechanism is provided.

The following code compiled just fine but did not at all act in the manner I expected:

BOOL CheckForThing(__in CObj *pObj, __in IFigMgr* pFigMgr, __in_opt LPCWSTR url)
{
    BOOL fCheck = FALSE;
    if (SubCheck(pObj))
    {
        ...

I’m calling SubCheck which looks like:

bool SubCheck(const CObj& obj);

Did you spot the bug? As you can see I should be passing in *pObj not pObj since the method takes a const CObj& not a CObj*. But then why does it compile?

It works because CObj has a constructor with all but one param with default values and CObj is derived from IUnknown:

CObj(__in_opt IUnknown * pUnkOuter, __in_opt LPCWSTR pszUrl = NULL);

Accordingly C++ uses this constructor as an implicit conversion operator. So instead of passing in my CObj, I end up creating a new CObj on the stack passing in the CObj I wanted as the outer object which has a number of issues.

The lesson is unless you really want this behavior, don't make constructors with all but 1 or 0 default parameters. If you need to do that consider using the 'explicit' keyword on the constructor.

More info about forcing single argument constructors to be explicit is available on stack overflow.

Shortly after joining the Internet Explorer team I got a bug from a PM on a popular Microsoft web server product that I'll leave unnamed (from now on UWS). The bug said that IE was handling empty path segments incorrectly by not removing them before resolving dotted path segments. For example UWS would do the following:

A.1. http://example.com/a/b//../
A.2. http://example.com/a/b/../
A.3. http://example.com/a/

In step 1 they are given a URI with dotted path segment and an empty path segment. In step 2 they remove the empty path segment, and in step 3 they resolve the dotted path segment. Whereas, given the same initial URI, IE would do the following:

B.1. http://example.com/a/b//../
B.2. http://example.com/a/b/

IE simply resolves the dotted path segment against the empty path segment and removes them both. So, how did I resolve this bug? As "By Design" of course!

The URI RFC allows path segments of zero length and does not assign them any special meaning. So generic user agents that intend to work on the web must not treat an empty path segment any different from a path segment with some text in it. In the case above IE is doing the correct thing.

That's the case for generic user agents, however servers may decide that a URI with an empty path segment returns the same resource as a the same URI without that empty path segment. Essentially they can decide to ignore empty path segments. Both IIS and Apache work this way and thus return the same resource for the following URIs:

http://exmaple.com/foo//bar///baz
http://example.com/foo/bar/baz

The issue for UWS is that it removes empty path segments before resolving dotted path segments. It must follow normal URI procedure before applying its own additional rules for empty path segments. Not doing that means they end up violating URI equivalency rules: URIs (A.1) and (B.2) are equivalent but UWS will not return the same resource for them.

A bug came up the other day involving markup containing <input type="image" src="http://example.com/.... I knew that "image" was a valid input type but it wasn't until that moment that I realized I didn't know what it did. Looking it up I found that it displays the specified image and when the user clicks on the image, the form is submitted with an additional two name value pairs: the x and y positions of the point at which the user clicked the image.

Take for example the following HTML:

<form action="http://example.com/">
<input type="image" name="foo" src="http://deletethis.net/dave/images/davebefore.jpg">
</form>

If the user clicks on the image, the browser will submit the form with a URI like the following:http://example.com/?foo.x=145&foo.y=124.

This seemed like an incredibly specific feature to be built directly into the language when this could instead be done with javascript. I looked a bit further and saw that its been in HTML since at least HTML2, which of course makes much more sense. Javascript barely existed at that point and sending off the user's click location in a form may have been the only way to do something interesting with that action.

The following code works fine. I have a ccomptr named resolvedUri and I want to update its hostname so I do the following:

        CreateIUriBuilder(resolvedUri, 0, 0, &builder);
        builder->SetHost(host);
        builder->CreateUri(0xFFFFFFFF, 0, 0, &resolvedUri);

But the following similar looking code has a bug:

    ResolveHost(resolvedUri, &resolvedUri);

The issue is that doing &resolvedUri gets the address of the pointer but also clears out the pointer due to the definition of my smart pointer class:

    operator T**()  
    {  
        T *ptrValue = mPtrValue;
        mPtrValue->Release();
        mPtrValue = NULL;
        return &ptrValue;
    }

In C++ there’s no guarantee about the order in which parameters for a function or method are evaluated. In the case above, &resolvedUri clears out the ccomptr before evaluating resolvedUri.Get() and so ResolveHostAlias gets a nullptr.

An interesting and related thread on stack overflow on undefined behavior in C++.