I've made a WPAD server Fiddler extension and in a fit of creativity I've named it: WPAD Server Fiddler Extension.
Of course you know about Fiddler, Eric's awesome HTTP debugger tool, the HTTP proxy that lets you inspect, visualize and modify the HTTP traffic that flows through it. And on the subject you've probably definitely heard of WPAD, the Web Proxy Auto Discovery protocol that allows web browsers like IE to use DHCP or DNS to automatically discover HTTP proxies on their network. While working on a particularly nasty WPAD bug towards the end of IE8 I really wished I had a way to see the WPAD requests and responses and modify PAC responses in Fiddler. Well the wishes of me of the past are now fulfilled by present day me as this Fiddler extension will respond to WPAD DHCP requests telling those clients (by default) that Fiddler is their proxy.
When I started working on this project I didn't really understand how DHCP worked especially with respect to WPAD. I won't bore you with my misconceptions: it works by having your one DHCP server on your network respond to regular DHCP requests as well as WPAD DHCP requests. And Windows I've found runs a DHCP client service (you can start/stop it via Start|Run|'services.msc', scroll to DHCP Client or via the command line with "net start/stop 'DHCP Client'") that caches DHCP server responses making it just slightly more difficult to test and debug my extension. If a Windows app uses the DHCP client APIs to ask for the WPAD option, this service will send out a DHCP request and take the first DHCP server response it gets. That means that if you're on a network with a DHCP server, my extension will be racing to respond to the client. If the DHCP server wins then the client ignores the WPAD response from my extension.
Various documents and tools I found useful while working on this:
I'm a big fan of the concept of registerProtocolHandler in HTML 5 and in FireFox 3, but not quite the implementation. From a high level, it allows web apps to register themselves as handlers of an URL scheme so for (the canonical) example, GMail can register for the mailto URL scheme. I like the concept:
registerProtocolHandler("info:lccn/{lccnID}", "htttp://www.librarything.com/search_works.php?q={lccnID}", "LibraryThing LCCN")
Windows allows for application protocols in which, through the registry, you specify a URL scheme and a command line to have that URL passed to your application. Its an easy way to hook a webbrowser up to your application. Anyone can read the doc above and then walk through the registry and pick out the application protocols but just from that info you can't tell what the application expects these URLs to look like. I did a bit of research on some of the application protocols I've seen which is listed below. Good places to look for information on URI schemes: Wikipedia URI scheme, and ESW Wiki UriSchemes.
| Scheme | Name | Notes |
|---|---|---|
| search-ms | Windows Search Protocol |
The search-ms application protocol is a convention for querying the Windows Search index. The protocol enables applications, like Microsoft Windows Explorer, to query the index with
parameter-value arguments, including property arguments, previously saved searches, Advanced Query Syntax, Natural Query Syntax, and language code identifiers (LCIDs) for both the Indexer and
the query itself. See the MSDN docs for search-ms for more info. Example: search-ms:query=food |
| Explorer.AssocProtocol.search-ms | ||
| OneNote | OneNote Protocol |
From the OneNote help: /hyperlink "pagetarget" - Starts OneNote and opens the page specified by the pagetarget parameter. To obtain the hyperlink for any page in a OneNote
notebook, right-click its page tab and then click Copy Hyperlink to this Page.Example: onenote:///\\GUMMO\Users\davris\Documents\OneNote%20Notebooks\OneNote%202007%20Guide\Getting%20Started%20with%20OneNote.one#section-id={692F45F5-A42A-415B-8C0D-39A10E88A30F}&end |
| callto | Callto Protocol |
ESW Wiki Info on callto Skype callto info NetMeeting callto info Example: callto://+12125551234 |
| itpc | iTunes Podcast |
Tells iTunes to subscribe to an indicated podcast. iTunes documentation. C:\Program Files\iTunes\iTunes.exe /url "%1" Example: itpc:http://www.npr.org/rss/podcast.php?id=35 |
| iTunes.AssocProtocol.itpc | ||
| pcast | ||
| iTunes.AssocProtocol.pcast | ||
| Magnet | Magnet URI | Magnet URL scheme described by Wikipedia. Magnet URLs identify a resource by a hash of that resource so that when used in P2P scenarios no central authority is necessary to create URIs for a resource. |
| mailto | Mail Protocol |
RFC 2368 - Mailto URL Scheme. Mailto Syntax Opens mail programs with new message with some parameters filled in, such as the to, from, subject, and body. Example: mailto:?to=david.risney@gmail.com&subject=test&body=Test of mailto syntax |
| WindowsMail.Url.Mailto | ||
| MMS | mms Protocol |
MSDN describes associated protocols. Wikipedia describes MMS. "C:\Program Files\Windows Media Player\wmplayer.exe" "%L" Also appears to be related to MMS cellphone messages: MMS IETF Draft. |
| WMP11.AssocProtocol.MMS | ||
| secondlife | [SecondLife] |
Opens SecondLife to the specified location, user, etc. SecondLife Wiki description of the URL scheme. "C:\Program Files\SecondLife\SecondLife.exe" -set SystemLanguage en-us -url "%1" Example: secondlife://ahern/128/128/128 |
| skype | Skype Protocol |
Open Skype to call a user or phone number. Skype's documentation Wikipedia summary of skype URL scheme "C:\Program Files\Skype\Phone\Skype.exe" "/uri:%l" Example: skype:+14035551111?call |
| skype-plugin | Skype Plugin Protocol Handler |
Something to do with adding plugins to skype? Maybe. "C:\Program Files\Skype\Plugin Manager\skypePM.exe" "/uri:%1" |
| svn | SVN Protocol |
Opens TortoiseSVN to browse the repository URL specified in the URL. C:\Program Files\TortoiseSVN\bin\TortoiseProc.exe /command:repobrowser /path:"%1" |
| svn+ssh | ||
| tsvn | ||
| webcal | Webcal Protocol |
Wikipedia describes webcal URL scheme. Webcal URL scheme description. A URL that starts with webcal:// points to an Internet location that contains a calendar in iCalendar format. "C:\Program Files\Windows Calendar\wincal.exe" /webcal "%1" Example: webcal://www.lightstalkers.org/LS.ics |
| WindowsCalendar.UrlWebcal.1 | ||
| zune | Zune Protocol |
Provides access to some Zune operations such as podcast subscription (via Zune Insider). "c:\Program Files\Zune\Zune.exe" -link:"%1" Example: zune://subscribe/?name=http://feeds.feedburner.com/wallstrip. |
| feed | Outlook Add RSS Feed |
Identify a resource that is a feed such as Atom or RSS. Implemented by Outlook to add the indicated feed to Outlook. Feed URI scheme pre-draft document "C:\PROGRA~2\MICROS~1\Office12\OUTLOOK.EXE" /share "%1" |
| im | IM Protocol |
RFC 3860 IM URI scheme description Like mailto but for instant messaging clients. Registered by Office Communicator but I was unable to get it to work as described in RFC 3860. "C:\Program Files (x86)\Microsoft Office Communicator\Communicator.exe" "%1" |
| tel | Tel Protocol |
RFC 5341 - tel URI scheme IANA assignment RFC 3966 - tel URI scheme description Call phone numbers via the tel URI scheme. Implemented by Office Communicator. "C:\Program Files (x86)\Microsoft Office Communicator\Communicator.exe" "%1" |
![['Neverending story' by Alexandre Duret-Lutz. A framed photo of books with the droste effect applied. Licensed under creative commons.]](http://farm1.static.flickr.com/90/252757784_3de44cbeb4_m_d.jpg)
Information about URI Fragments, the portion of URIs
that follow the '#' at the end and that are used to navigate within a document, is scattered throughout various documents which I usually have to hunt down. Instead I'll link to them all here.
Definitions. Fragments are defined in the URI RFC which states that they're used to identify a secondary resource that is related to the primary resource identified by the URI as a subset of the primary, a view of the primary, or some other resource described by the primary. The interpretation of a fragment is based on the mime type of the primary resource. Tim Berners-Lee notes that determining fragment meaning from mime type is a problem because a single URI may contain a single fragment, however over HTTP a single URI can result in the same logical resource represented in different mime types. So there's one fragment but multiple mime types and so multiple interpretations of the one fragment. The URI RFC says that if an author has a single resource available in multiple mime types then the author must ensure that the various representations of a single resource must all resolve fragments to the same logical secondary resource. Depending on which mime types you're dealing with this is either not easy or not possible.
HTTP. In HTTP when URIs are used, the fragment is not included. The General Syntax section of the HTTP standard says it uses the definitions of 'URI-reference' (which includes the fragment), 'absoluteURI', and 'relativeURI' (which don't include the fragment) from the URI RFC. However, the 'URI-reference' term doesn't actually appear in the BNF for the protocol. Accordingly the headers like 'Request-URI', 'Content-Location', 'Location', and 'Referer' which include URIs are defined with 'absoluteURI' or 'relativeURI' and don't include the fragment. This is in keeping with the original fragment definition which says that the fragment is used as a view of the original resource and consequently only needed for resolution on the client. Additionally, the URI RFC explicitly notes that not including the fragment is a privacy feature such that page authors won't be able to stop clients from viewing whatever fragments the client chooses. This seems like an odd claim given that if the author wanted to selectively restrict access to portions of documents there are other options for them like breaking out the parts of a single resource to which the author wishes to restrict access into separate resources.
HTML. In HTML, the HTML mime type RFC defines HTML's fragment use which consists of fragments referring to elements with a corresponding 'id' attribute or one of a particular set of elements with a corresponding 'name' attribute. The HTML spec discusses fragment use additionally noting that the names and ids must be unique in the document and that they must consist of only US-ASCII characters. The ID and NAME attributes are further restricted in section 6 to only consist of alphanumerics, the hyphen, period, colon, and underscore. This is a subset of the characters allowed in the URI fragment so no encoding is discussed since technically its not needed. However, practically speaking, browsers like FireFox and Internet Explorer allow for names and ids containing characters outside of the defined set including characters that must be percent-encoded to appear in a URI fragment. The interpretation of percent-encoded characters in fragments for HTML documents is not consistent across browsers (or in some cases within the same browser) especially for the percent-encoded percent.
Text. Text/plain recently got a fragment definition that allows fragments to refer to particular lines or characters within a text document. The scheme no longer includes regular expressions, which disappointed me at first, but in retrospect is probably good idea for increasing the adoption of this fragment scheme and for avoiding the potential for ubiquitous DoS via regex. One of the authors also notes this on his blog. I look forward to the day when this scheme is widely implemented.
XML. XML has the XPointer framework to define its fragment structure as noted by the XML mime type definition. XPointer consists of a general scheme that contains subschemes that identify a subset of an XML document. Its too bad such a thing wasn't adopted for URI fragments in general to solve the problem of a single resource with multiple mime type representations. I wrote more about XPointer when I worked on hacking XPointer into IE.
SVG and MPEG. Through the Media Fragments Working Group I found a couple more fragment scheme definitions. SVG's fragment scheme is defined in the SVG documentation and looks similar to XML's. MPEG has one defined but I could only find it as an ISO document "Text of ISO/IEC FCD 21000-17 MPEG-12 FID" and not as an RFC which is a little disturbing.
AJAX. AJAX websites have used fragments as an escape hatch for two issues that I've seen. The first is getting a unique URL for versions of a page that are produced on the client by script. The fragment may be changed by script without forcing the page to reload. This goes outside the rules of the standards by using HTML fragments in a fashion not called out by the HTML spec. but it does seem to be inline with the spirit of the fragment in that it is a subview of the original resource and interpretted client side. The other hack-ier use of the fragment in AJAX is for cross domain communication. The basic idea is that different frames or windows may not communicate in normal fashions if they have different domains but they can view each other's URLs and accordingly can change their own fragments in order to send a message out to those who know where to look. IMO this is not inline with the spirit of the fragment but is rather a cool hack.
IPv6 address syntax consists of 8 groupings of colon delimited 16-bit hex values making up the 128-bit address. An optional double colon
can replace any consecutive sequence of 0 valued hex values. For example the following is a valid IPv6 address: fe80::2c02:db79
Some IPv6 addresses aren't global and in those cases need a scope ID to describe their context. These get a '%' followed by the scope ID.
For example the previous example with a scope ID of '8' would be: fe80::2c02:db79%8
IPv6 addresses in URIs may appear in the host section of a URI as long as they're enclosed by square brackets. For example:
http://[fe80::2c02:db79]/. The RFC explicitly notes that there isn't a way to add a scope ID to the IPv6 address in a URI. However a draft document describes adding
scope IDs to IPv6 addresses in URIs. The draft document uses the IPvFuture production from the URI RFC with a 'v1' to add a new
hostname syntax and a '+' instead of a '%' for delimiting the scope id. For example: http://[v1.fe80::2c02:db79+8]/. However, this is still a draft document, not a final
standard, and I don't know of any system that works this way.
In Windows XPSP2 the IPv6 stack is available but disabled by default. To enable the IPv6 stack, at a command prompt run 'netsh interface ipv6 install'. In Vista IPv6 is the on by default and cannot be turned off, while the IPv4 stack is optional and may be turned off by a command similar to the previous.
Once you have IPv6 on in your OS you can turn on IPv6 for IIS6 or just use IIS7. The address ::1 refers to the local machine.
In some places in Windows like UNC paths, IPv6 addresses aren't allowed. In those cases you can use a Vista DNS IPv6 hack that lives in the OS
name resolution stack that transforms particularly crafted names into IPv6 addresses. Take your IPv6 address, replace the ':'s with '-'s and the '%' with an 's' and then append '.ipv6-literal.net'
to the end. For example: fe80--2c02-db79s8.ipv6-literal.net. That name will resolve to the same example I've been using in Vista. This transformation occurs inside the system's local
name resolution stack so no DNS servers are involved, although Microsoft does own the ipv6-literal.net domain name.
MSDN describes IPv6 addresses in URIs in Windows and I've described IPv6 addresses in URIs in IE7. File URIs in
IE7 don't support IPv6 addresses. If you want to put a scope ID in a URI in IE7 you use a '%25' to delimit the scope ID and due to a bug you must have at least two digits in your scope ID. So,
to take the previous example: http://[fe80::2c02:db79%2508]/. Note that its 08 rather than just 8.
I tried searching for information on this and I've found other delicious users doing the same thing,
but nothing about the tag itself. If you know any information especially official information from del.icio.us itself please post links in reply to this post. So without further preface here's what
I've learned about the del.icio.us shortcut tag.
Some rights reserved