diff page 3 - Dave's Blog

Stripe is running a web security capture the flag - a series of increasingly difficult web security exploit challenges. I've finished it and had a lot of fun. Working on a web browser I knew the theory of these various web based attacks, but this was my first chance to put theory into practice with:

• No adverse consequences
• Knowledge that there is a fun security exploit to find
• Access to the server side source code

Here's a blog post on the CTF behind the scenes setup which has many impressive features including phantom users that can be XSS/CSRF'ed.

I'll have another post on my difficulties and answers for the CTF levels after the contest is over on Wed, but if you're looking for hints, try out the CTF chatroom or the level specific CTF chatroom.

Use of my old Hotmail account has really snuck up on me as I end up caring more and more about all of the services with which it is associated. The last straw is Windows 8 login, but previous straws include Xbox, Zune, SkyDrive, and my Windows 7 Phone. I like the features and sync'ing associated with the Windows Live ID, but I don't like my old, spam filled, hotmail email address on the Live ID account.

A coworker told me about creating a Live ID from a custom domain, which sounded like just the ticket for me. Following the instructions above I was able to create a new deletethis.net Live ID but the next step of actually using this new Live ID was much more difficult. My first hope was there would be some way to link my new and old Live IDs so as to make them interchangeable. As it turns out there is a way to link Live IDs but all that does is make it easy to switch between accounts on Live Mail, SkyDrive and some other webpages.

Instead one must change over each service or start over depending on the service:

Xbox

In the Xbox 360 system menu you can change the Live ID associated with your gamertag. This worked fine for me and I got an email telling me about the transfer of my Microsoft Points.

Zune

There's no way to do this for the Zune specifically, however changing over your Xbox account also transfers over all your Zune purchased content. I don't have a Zune Pass so I can't confirm that, but all of my previously purchased television shows transferred over successfully.

Windows 7 Phone

To change the main Live ID associated with your phone, reset your phone to factory default and start over. All purchased applications are lost. Had I purchased any applications I would have been pissed, but instead I was just irritated that I had to reset my phone.

Mail

I don't use my Hotmail account for anything and it only sits and collects spam. Accordingly I didn't attempt switching this over.

SkyDrive

I didn't have much in my SkyDrive account. I downloaded all files as a zip and then manually uploaded them to the new account.

With Facebook changing its privacy policy and settings so frequently and just generally the huge amount of social sites out there, for many of us it is far too late to ensure our name doesn't show up with unfortunate results in web searches. Information is too easily copyable and archive-able to make removing these results a viable option, so clearly the solution is to create more data.

Create fake profiles on Facebook using your name but with a different photo, different date of birth, and different hometown. Create enough doppelgangers to add noise to the search results for your name. And have them share embarrassing stories on their blogs. The goal is to ensure that the din of your alternates drowns out anything embarrassing showing up for you.

Although it will look suspicious if you're the only name on Google with such chaff. So clearly you must also do this for your friends and family. Really you'll be doing them a favor.

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Getting into the more subtle levels of URI percent-encoding ignorance, folks try to apply their knowledge of percent-encoding to URIs as a whole producing the concepts escaped URIs and unescaped URIs. However there are no such things - URIs themselves aren't percent-encoded or decoded but rather contain characters that are percent-encoded or decoded. Applying percent-encoding or decoding to a URI as a whole produces a new and non-equivalent URI.

Instead of lingering on the incorrect concepts we'll just cover the correct ones: there's raw unencoded data, non-normal form URIs and normal form URIs. For example:

1. http://example.com/%74%68%65%3F%70%61%74%68?query
2. http://example.com/the%3Fpath?query
3. "http", "example.com", "the?path", "query"

In the above (A) is not an 'encoded URI' but rather a non-normal form URI. The characters of 'the' and 'path' are percent-encoded but as unreserved characters specific in the RFC should not be encoded. In the normal form of the URI (B) the characters are decoded. But (B) is not a 'decoded URI' -- it still has an encoded '?' in it because that's a reserved character which by the RFC holds different meaning when appearing decoded versus encoded. Specifically in this case, it appears encoded which means it is data -- a literal '?' that appears as part of the path segment. This is as opposed to the decoded '?' that appears in the URI which is not part of the path but rather the delimiter to the query.

Usually when developers talk about decoding the URI what they really want is the raw data from the URI. The raw decoded data is (C) above. The only thing to note beyond what's covered already is that to obtain the decoded data one must parse the URI before percent decoding all percent-encoded octets.

Of course the exception here is when a URI is the raw data. In this case you must percent-encode the URI to have it appear in another URI. More on percent-encoding while constructing URIs later.

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping). The basest ignorance is with respect to the mere existence of percent-encoding. Percents in URIs are special: they always represent the start of a percent-encoded octet. That is to say, a percent is always followed by two hex digits that represents a value between 0 and 255 and doesn't show up in a URI otherwise.

The IPv6 textual syntax for scoped addresses uses the '%' to delimit the zone ID from the rest of the address. When it came time to define how to represent scoped IPv6 addresses in URIs there were two camps: Folks who wanted to use the IPv6 format as is in the URI, and those who wanted to encode or replace the '%' with a different character. The resulting thread was more lively than what shows up on the IETF URI discussion mailing list. Ultimately we went with a percent-encoded '%' which means the percent maintains its special status and singular purpose.

Shortly after joining the Internet Explorer team I got a bug from a PM on a popular Microsoft web server product that I'll leave unnamed (from now on UWS). The bug said that IE was handling empty path segments incorrectly by not removing them before resolving dotted path segments. For example UWS would do the following:

A.1. http://example.com/a/b//../
A.2. http://example.com/a/b/../
A.3. http://example.com/a/

In step 1 they are given a URI with dotted path segment and an empty path segment. In step 2 they remove the empty path segment, and in step 3 they resolve the dotted path segment. Whereas, given the same initial URI, IE would do the following:

B.1. http://example.com/a/b//../
B.2. http://example.com/a/b/

IE simply resolves the dotted path segment against the empty path segment and removes them both. So, how did I resolve this bug? As "By Design" of course!

The URI RFC allows path segments of zero length and does not assign them any special meaning. So generic user agents that intend to work on the web must not treat an empty path segment any different from a path segment with some text in it. In the case above IE is doing the correct thing.

That's the case for generic user agents, however servers may decide that a URI with an empty path segment returns the same resource as a the same URI without that empty path segment. Essentially they can decide to ignore empty path segments. Both IIS and Apache work this way and thus return the same resource for the following URIs:

http://exmaple.com/foo//bar///baz
http://example.com/foo/bar/baz

The issue for UWS is that it removes empty path segments before resolving dotted path segments. It must follow normal URI procedure before applying its own additional rules for empty path segments. Not doing that means they end up violating URI equivalency rules: URIs (A.1) and (B.2) are equivalent but UWS will not return the same resource for them.

I had previously replaced my use of Delicious with Google Reader. Delicious had a number of issues during their switch over from Yahoo to the new owners and I was eventually fed up enough to remove it from daily use. I used Delicious to do the following things:

• Create a list of things to read later
• Save things to read again in the future
• Search through things I read and enjoyed (esp via tags)
• Annotate and share things on my blog

I realized that since I did most of my web browsing in Google Reader now anyway I may as well make use of its features. I star things to note I want to read it later or save to read again later. I can annotate with notes in Google Reader and I can share items to my web site by way of the shared items feed. Additionally for when I'm not in Google Reader there's a bookmarklet to add an arbitrary web site as a shared item in Google Reader.

Of course I wrote this and switched over about 1 week before Google removed the sharing feature from Google Reader. I'm irritated but in practice it forced me to find a different option which has worked out mostly better. New blog post coming soon about that...

I wanted to ensure that my switch statement in my implementation of IInternetSecurityManager::ProcessURLAction had a case for every possible documented URLACTION. I wrote the following short command line sequence to see the list of all URLACTIONs in the SDK header file not found in my source file:

grep URLACTION urlmon.idl | sed 's/.*\(URLACTION[a-zA-Z0-9_]*\).*/\1/g;' | sort | uniq > allURLACTIONs.txt
grep URLACTION MySecurityManager.cpp | sed 's/.*\(URLACTION[a-zA-Z0-9_]*\).*/\1/g;' | sort | uniq > myURLACTIONs.txt
comm -23 allURLACTIONs.txt myURLACTIONs.txt

I'm not a sed expert so I had to read the sed documentation, and I heard about comm from Kris Kowal's blog which happilly was in the Win32 GNU tools pack I already run.

But in my effort to learn and use PowerShell I found the following similar command line:

diff 
(more urlmon.idl | %{ if ($_ -cmatch "URLACTION[a-zA-Z0-9_]*") { $matches[0] } } | sort -uniq) 
(more MySecurityManager.cpp | %{ if ($_ -cmatch "URLACTION[a-zA-Z0-9_]*") { $matches[0] } } | sort -uniq)

In the PowerShell version I can skip the temporary files which is nice. 'diff' is mapped to 'compare-object' which seems similar to comm but with no parameters to filter out the different streams (although this could be done more verbosely with the ?{ } filter syntax). In PowerShell uniq functionality is built into sort. The builtin -cmatch operator (c is for case sensitive) to do regexp is nice plus the side effect of generating the $matches variable with the regexp results.

Working on GeolocMock it took me a bit to realize why my HTML could use the W3C Geolocation API in IE9 but not in my WebBrowser control in my .NET application. Eventually I realized that I was getting the wrong IE doc mode. Reading this old More IE8 Extensibility Improvements IE blog post from the IE blog I found the issue is that for app compat the WebOC picks older doc modes but an app hosting the WebOC can set a regkey to get different doc modes. The IE9 mode isn't listed in that article but I took a guess based on the values there and the decimal value 9999 gets my app IE9 mode. The following is the code I run in my application to set its regkey so that my app can get the IE9 doc mode and use the geolocation API.

        static private void UseIE9DocMode()
        {
            RegistryKey key = null;
            try
            {
                key = Registry.CurrentUser.OpenSubKey("Software\\Microsoft\\Internet Explorer\\Main\\FeatureControl\\FEATURE_BROWSER_EMULATION", true);
            }
            catch (Exception)
            {
                key = Registry.CurrentUser.CreateSubKey("Software\\Microsoft\\Internet Explorer\\Main\\FeatureControl\\FEATURE_BROWSER_EMULATION");
            }
            key.SetValue(System.Diagnostics.Process.GetCurrentProcess().MainModule.ModuleName, 9999, RegistryValueKind.DWord);
            key.Close();
        }

I've made two simple command line tools related to the console window and Win7 jump lists. The source is available for both but neither is much more than the sort of samples you'd find on MSDN =).

• SetAppUserModelId
• SetJumpList

SetAppUserModelId lets you change the Application User Model ID for the current console window. The AppUserModelId is the value Win7 uses to group together icons on the task bar and is what the task bar's jump lists are associated with. The tool lets you change that as well as the icon and name that appear in the task bar for the window, and the command to launch if the user attempts to re-launch the application from its task bar icon.

SetJumpList lets you set the jump list associated with a particular AppUserModelId. You pass the AppUserModelId as the only parameter and then in its standard input you give it lines specifying items that should appear in the jump list and what to execute when those items are picked.

I put these together to make my build environment easier to deal with at work. I have to deal with multiple enlistments in many different branches and so I wrote a simple script around these two tools to group my build windows by branch name in the task bar, and to add the history of commands I've used to launch the build environment console windows to the jump list of each.