URI functions in Windows Store Applications

2013 Jul 25, 1:00

Summary

The Modern SDK contains some URI related functionality as do libraries available in particular projection languages. Unfortunately, collectively these APIs do not cover all scenarios in all languages. Specifically, JavaScript and C++ have no URI building APIs, and C++ additionally has no percent-encoding/decoding APIs.

	WinRT (JS and C++)	JS Only	C++ Only	.NET Only
Parse	Windows.Foundation.Uri		WsDecodeUrl*	System.Uri
Build			WsEncodeUrl*	System.UriBuilder
Normalize	Windows.Foundation.Uri			System.Uri
Equality	Windows.Foundation.Uri. Equals			System.Uri.Compare
Relative resolution	Windows.Foundation.Uri. CombineUri			System.Uri
Encode data for including in URI property	Windows.Foundation.Uri. EscapeComponent	encodeURIComponent	WsEncodeUrl*	System.Uri. EscapeDataString
Decode data extracted from URI property	Windows.Foundation.Uri. UnescapeComponent	decodeURIComponent	WsDecodeUrl*	System.Uri. UnescapeDataString
Build Query
Parse Query	Windows.Foundation. Uri.QueryParsed / Windows.Foundation. WwwFormUrlDecoder*

The Windows.Foudnation.Uri type is not projected into .NET modern applications. Instead those applications use System.Uri and the platform ensures that it is correctly converted back and forth between Windows.Foundation.Uri as appropriate. Accordingly the column marked WinRT above is applicable to JS and C++ modern applications but not .NET modern applications. The only entries above applicable to .NET are the .NET Only column and the WwwFormUrlDecoder in the bottom left which is available to .NET.

Scenarios

Parse

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS, and by System.Uri in .NET.

Parsing a URI pulls it apart into its basic components without decoding or otherwise modifying the contents.

var uri = new Windows.Foundation.Uri("http://example.com/path%20segment1/path%20segment2?key1=value1&key2=value2");

console.log(uri.path);// /path%20segment1/path%20segment2

WsDecodeUrl (C++)

WsDecodeUrl is not suitable for general purpose URI parsing. Use Windows.Foundation.Uri instead.

Build (C#)

URI building is only available in C# via System.UriBuilder.

URI building is the inverse of URI parsing: URI building allows the developer to specify the value of basic components of a URI and the API assembles them into a URI.

To work around the lack of a URI building API developers will likely concatenate strings to form their URIs. This can lead to injection bugs if they don’t validate or encode their input properly, but if based on trusted or known input is unlikely to have issues.

Uri originalUri = new Uri("http://example.com/path1/?query");

UriBuilder uriBuilder = new UriBuilder(originalUri);

uriBuilder.Path = "/path2/";

Uri newUri = uriBuilder.Uri; // http://example.com/path2/?query

WsEncodeUrl (C++)

WsEncodeUrl, in addition to building a URI from components also does some encoding. It encodes non-US-ASCII characters as UTF8, the percent, and a subset of gen-delims based on the URI property: all :/?#[]@ are percent-encoded except :/@ in the path and :/?@ in query and fragment.

Accordingly, WsEncodeUrl is not suitable for general purpose URI building. It is acceptable to use in the following cases:

- You’re building a URI out of non-encoded URI properties and don’t care about the difference between encoded and decoded characters. For instance you’re the only one consuming the URI and you uniformly decode URI properties when consuming – for instance using WsDecodeUrl to consume the URI.

- You’re building a URI with URI properties that don’t contain any of the characters that WsEncodeUrl encodes.

Normalize

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET. Normalization is applied during construction of the Uri object.

URI normalization is the application of URI normalization rules (including DNS normalization, IDN normalization, percent-encoding normalization, etc.) to the input URI.

var normalizedUri = new Windows.Foundation.Uri("HTTP://EXAMPLE.COM/p%61th foo/");

console.log(normalizedUri.absoluteUri); // http://example.com/path%20foo/

This is modulo Win8 812823 in which the Windows.Foundation.Uri.AbsoluteUri property returns a normalized IRI not a normalized URI. This bug does not affect System.Uri.AbsoluteUri which returns a normalized URI.

Equality

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET.

URI equality determines if two URIs are equal or not necessarily equal.

var uri1 = new Windows.Foundation.Uri("HTTP://EXAMPLE.COM/p%61th foo/"),

uri2 = new Windows.Foundation.Uri("http://example.com/path%20foo/");

console.log(uri1.equals(uri2)); // true

Relative resolution

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET

Relative resolution is a function that given an absolute URI A and a relative URI B, produces a new absolute URI C. C is the combination of A and B in which the basic components specified in B override or combine with those in A under rules specified in RFC 3986.

var baseUri = new Windows.Foundation.Uri("http://example.com/index.html"),

relativeUri = "/path?query#fragment",

absoluteUri = baseUri.combineUri(relativeUri);

console.log(baseUri.absoluteUri); // http://example.com/index.html

console.log(absoluteUri.absoluteUri); // http://example.com/path?query#fragment

Encode data for including in URI property

This functionality is available in JavaScript via encodeURIComponent and in C# via System.Uri.EscapeDataString. Although the two methods mentioned above will suffice for this purpose, they do not perform exactly the same operation.

Additionally we now have Windows.Foundation.Uri.EscapeComponent in WinRT, which is available in JavaScript and C++ (not C# since it doesn’t have access to Windows.Foundation.Uri). This is also slightly different from the previously mentioned mechanisms but works best for this purpose.

Encoding data for inclusion in a URI property is necessary when constructing a URI from data. In all the above cases the developer is dealing with a URI or substrings of a URI and so the strings are all encoded as appropriate. For instance, in the parsing example the path contains “path%20segment1” and not “path segment1”. To construct a URI one must first construct the basic components of the URI which involves encoding the data. For example, if one wanted to include “path segment / example” in the path of a URI, one must percent-encode the ‘ ‘ since it is not allowed in a URI, as well as the ‘/’ since although it is allowed, it is a delimiter and won’t be interpreted as data unless encoded.

If a developer does not have this API provided they can write it themselves. Percent-encoding methods appear simple to write, but the difficult part is getting the set of characters to encode correct, as well as handling non-US-ASCII characters.

var uri = new Windows.Foundation.Uri("http://example.com" +

"/" + Windows.Foundation.Uri.escapeComponent("path segment / example") +

"?key=" + Windows.Foundation.Uri.escapeComponent("=&?#"));

console.log(uri.absoluteUri); // http://example.com/path%20segment%20%2F%20example?key=%3D%26%3F%23

WsEncodeUrl (C++)

In addition to building a URI from components, WsEncodeUrl also percent-encodes some characters. However the API is not recommend for this scenario given the particular set of characters that are encoded and the convoluted nature in which a developer would have to use this API in order to use it for this purpose.

There are no general purpose scenarios for which the characters WsEncodeUrl encodes make sense: encode the %, encode a subset of gen-delims but not also encode the sub-delims. For instance this could not replace encodeURIComponent in a C++ version of the following code snippet since if ‘value’ contained ‘&’ or ‘=’ (both sub-delims) they wouldn’t be encoded and would be confused for delimiters in the name value pairs in the query:

"http://example.com/?key=" + Windows.Foundation.Uri.escapeComponent(value)

Since WsEncodeUrl produces a string URI, to obtain the property they want to encode they’d need to parse the resulting URI. WsDecodeUrl won’t work because it decodes the property but Windows.Foundation.Uri doesn’t decode. Accordingly the developer could run their string through WsEncodeUrl then Windows.Foundation.Uri to extract the property.

Decode data extracted from URI property

This functionality is available in JavaScript via decodeURIComponent and in C# via System.Uri.UnescapeDataString. Although the two methods mentioned above will suffice for this purpose, they do not perform exactly the same operation.

Additionally we now also have Windows.Foundation.Uri.UnescapeComponent in WinRT, which is available in JavaScript and C++ (not C# since it doesn’t have access to Windows.Foundation.Uri). This is also slightly different from the previously mentioned mechanisms but works best for this purpose.

Decoding is necessary when extracting data from a parsed URI property. For example, if a URI query contains a series of name and value pairs delimited by ‘=’ between names and values, and by ‘&’ between pairs, one must first parse the query into name and value entries and then decode the values. It is necessary to make this an extra step separate from parsing the URI property so that sub-delimiters (in this case ‘&’ and ‘=’) that are encoded will be interpreted as data, and those that are decoded will be interpreted as delimiters.

If a developer does not have this API provided they can write it themselves. Percent-decoding methods appear simple to write, but have some tricky parts including correctly handling non-US-ASCII, and remembering not to decode .

In the following example, note that if unescapeComponent were called first, the encoded ‘&’ and ‘=’ would be decoded and interfere with the parsing of the name value pairs in the query.

var uri = new Windows.Foundation.Uri("http://example.com/?foo=bar&array=%5B%27%E3%84%93%27%2C%27%26%27%2C%27%3D%27%2C%27%23%27%5D");

uri.query.substr(1).split("&").forEach(

function (keyValueString) {

var keyValue = keyValueString.split("=");

console.log(Windows.Foundation.Uri.unescapeComponent(keyValue[0]) + ": " + Windows.Foundation.Uri.unescapeComponent(keyValue[1]));

// foo: bar

// array: ['ㄓ','&','=','#']

});

WsDecodeUrl (C++)

Since WsDecodeUrl decodes all percent-encoded octets it could be used for general purpose percent-decoding but it takes a URI so would require the dev to construct a stub URI around the string they want to decode. For example they could prefix “http:///#” to their string, run it through WsDecodeUrl and then extract the fragment property. It is convoluted but will work correctly.

Parse Query

The query of a URI is often encoded as application/x-www-form-urlencoded which is percent-encoded name value pairs delimited by ‘&’ between pairs and ‘=’ between corresponding names and values.

In WinRT we have a class to parse this form of encoding using Windows.Foundation.WwwFormUrlDecoder. The queryParsed property on the Windows.Foundation.Uri class is of this type and created with the query of its Uri:

var uri = Windows.Foundation.Uri("http://example.com/?foo=bar&array=%5B%27%E3%84%93%27%2C%27%26%27%2C%27%3D%27%2C%27%23%27%5D");

uri.queryParsed.forEach(

function (pair) {

console.log("name: " + pair.name + ", value: " + pair.value);

// name: foo, value: bar

// name: array, value: ['ㄓ','&','=','#']

});

console.log(uri.queryParsed.getFirstValueByName("array")); // ['ㄓ','&','=','#']

The QueryParsed property is only on Windows.Foundation.Uri and not System.Uri and accordingly is not available in .NET. However the Windows.Foundation.WwwFormUrlDecoder class is available in C# and can be used manually:

Uri uri = new Uri("http://example.com/?foo=bar&array=%5B%27%E3%84%93%27%2C%27%26%27%2C%27%3D%27%2C%27%23%27%5D");

WwwFormUrlDecoder decoder = new WwwFormUrlDecoder(uri.Query);

foreach (IWwwFormUrlDecoderEntry entry in decoder)

{

System.Diagnostics.Debug.WriteLine("name: " + entry.Name + ", value: " + entry.Value);

// name: foo, value: bar

// name: array, value: ['ㄓ','&','=','#']

}

Build Query

To build a query of name value pairs encoded as application/x-www-form-urlencoded there is no WinRT API to do this directly. Instead a developer must do this manually making use of the code described in “Encode data for including in URI property”.

In terms of public releases, this property is only in the RC and later builds.

For example in JavaScript a developer may write:

var uri = new Windows.Foundation.Uri("http://example.com/"),

query = "?" + Windows.Foundation.Uri.escapeComponent("array") + "=" + Windows.Foundation.Uri.escapeComponent("['ㄓ','&','=','#']");

console.log(uri.combine(new Windows.Foundation.Uri(query)).absoluteUri); // http://example.com/?array=%5B'%E3%84%93'%2C'%26'%2C'%3D'%2C'%23'%5D

c# c++ javascript technical uri windows windows-runtime windows-store

laughingsquid: Arrested Development Season 4 Trailer Brings...

2013 May 13, 2:47

laughingsquid:

Arrested Development Season 4 Trailer Brings Back the Dysfunctional Bluth Family

Come on!

humor trailer arrested-development awesome

Learn from Haskell - Functional, Reusable JavaScript (github.com)

2012 Feb 21, 3:32

technical programming functional haskell javascript

Command line for finding missing URLACTIONs

2011 May 28, 11:00

I wanted to ensure that my switch statement in my implementation of IInternetSecurityManager::ProcessURLAction had a case for every possible documented URLACTION. I wrote the following short command line sequence to see the list of all URLACTIONs in the SDK header file not found in my source file:

grep URLACTION urlmon.idl | sed 's/.*\(URLACTION[a-zA-Z0-9_]*\).*/\1/g;' | sort | uniq > allURLACTIONs.txt
grep URLACTION MySecurityManager.cpp | sed 's/.*\(URLACTION[a-zA-Z0-9_]*\).*/\1/g;' | sort | uniq > myURLACTIONs.txt
comm -23 allURLACTIONs.txt myURLACTIONs.txt

I'm not a sed expert so I had to read the sed documentation, and I heard about comm from Kris Kowal's blog which happilly was in the Win32 GNU tools pack I already run.

But in my effort to learn and use PowerShell I found the following similar command line:

diff 
(more urlmon.idl | %{ if ($_ -cmatch "URLACTION[a-zA-Z0-9_]*") { $matches[0] } } | sort -uniq) 
(more MySecurityManager.cpp | %{ if ($_ -cmatch "URLACTION[a-zA-Z0-9_]*") { $matches[0] } } | sort -uniq)

In the PowerShell version I can skip the temporary files which is nice. 'diff' is mapped to 'compare-object' which seems similar to comm but with no parameters to filter out the different streams (although this could be done more verbosely with the ?{ } filter syntax). In PowerShell uniq functionality is built into sort. The builtin -cmatch operator (c is for case sensitive) to do regexp is nice plus the side effect of generating the $matches variable with the regexp results.

powershell tool cli technical command line

FBJS - Facebook Developer Wiki

2010 May 6, 7:25Another subset of javascript and DOM access to make a sandbox: "FBJS is Facebook's solution for developers who want to use JavaScript in their Facebook applications. We built FBJS to empower developers with all the functionality they need, and to protect our users' privacy at the same time."

sandbox web browser facebook html javascript technical security web-sandbox

Where's Walden? » Whole-text DOM functionality and Acid3 redux

2010 Mar 18, 7:15This article describes the largest problem with the Acid3 test: "Acid3 often didn’t test things web authors wanted, but instead it tested things that were broken or not implemented regardless whether anyone truly cared."

acid3 web browser html dom test technical

GL10 (Java Binding for the OpenGL ES(R) API)

2009 Dec 25, 6:15"The GL10 interface contains the Java(TM) programming language bindings for OpenGL(R) ES 1.0 core functionality."

programming java opengl android cellphone technical reference 3d

YouTube - Google Chrome OS Demo

2009 Nov 20, 7:20I think I'm stuck on the first part of the Ars review "so it has taken the netbook, which was already a crippled notebook, and crippled it even further by removing a ton of flexibility and functionality". Still conceptually I like the idea and hope they figure out all their use cases.

google chrome video os web browser technical

Flickr Visual Search in IE8

2009 Apr 10, 9:48

A while ago I promised to say how an xsltproc Meddler script would be useful and the general answer is its useful for hooking up a client application that wants data from the web in a particular XML format and the data is available on the web but in another XML format. The specific case for this post is a Flickr Search service that includes IE8 Visual Search Suggestions. IE8 wants the Visual Search Suggestions XML format and Flickr gives out search data in their Flickr web API XML format.

So I wrote an XSLT to convert from Flickr Search XML to Visual Suggestions XML and used my xsltproc Meddler script to actually apply this xslt.

After getting this all working I've placed the result in two places: (1) I've updated the xsltproc Meddler script to include this XSLT and an XML file to install it as a search provider - although you'll need to edit the XML to include your own Flickr API key. (2) I've created a service for this so you can just install the Flickr search provider if you're interested in having the functionality and don't care about the implementation. Additionally, to the search provider I've added accelerator preview support to show the Flickr slideshow which I think looks snazzy.

Doing a quick search for this it looks like there's at least one other such implementation, but mine has the distinction of being done through XSLT which I provide, updated XML namespaces to work with the released version of IE8, and I made it so you know its good.

meddler xml ie8 xslt flickr technical boring search suggestions

G1 Android Phone

2008 Nov 9, 11:29

I finally replaced my old regular cell-phone which was literally being held together by a rubber band with a fancy new G1, my first Internet accessible phone.

I had to call the T-Mobile support line to get data added to my plan and the person helping me was disconcertingly friendly. She asked about my weekend plans and so I felt compelled to ask her the same. Her plans involved replacing her video card so she could get back to World of Warcraft and do I enjoy computer gaming? I couldn't tell if she was genuine or if she was signing me up for magazines.

I was with Sarah in her new car, trying out the phone's GPS functionality via Google Maps while she drove. I switched to Street View and happened to find my car. It was a weird feeling, kind of like those Google conspiracy videos.

The phone runs Google's open source OS and I really enjoy the application API. Its all in Java and URIs and mime-types are sort of basics. Rather than invoking the builtin item picker control directly you invoke an 'intent' specifying the URI of your list of items, a mime-type describing the type of items in the list, and an action 'PICK' and whatever is registered as the picker on the system pops up and lets the user pick from that list. The same goes if you want to 'EDIT' an image, or 'VIEW' an mp3.

I wanted to replace the Google search box gadget that appears on the home screen with my own search box widget that uses OpenSearch descriptors but apparently in the current API you can't make home screen gadgets without changing parts of the OS. My other desired application is something to replace this GPS photo tracker device by recording my location to a file and an additional program on my computer to apply those locations to photos.

tmobile personal api phone technical g1 android google

Mozilla Labs launches geolocation extension

2008 Oct 13, 2:21Neat geolocation API for web apps: "Mozilla Labs has announced the availability of Geode, an experimental Firefox extension that implements the W3C Geolocation Specification. Geode provides an early preview of the same location-aware functionality that will be included in both Fennec and Firefox 3.1."

geolocation geo w3c mozilla javascript web

Disemvowelment and Reemvowelment Tools

2008 Oct 3, 5:29I thought the disemvowelment of trolls was a pretty funny punishment -- much better than simply removing the comment: "Disemvowelment is - obviously enough - the act of removing the vowels from a passage of text, as well as a pun on the word 'disembowelling'. A number of blogs and websites do this to offensive text which has been placed in their 'comments' section. ... This site exists because I couldn't resists the challenge of trying to re-emvowel disemvowelled text. This is a challenging task, as the disemvowelled word 'dg' may well have been 'dog', but also 'dig', 'dug', 'doge', diego' and so on. I have a first cut of this functionality at the re-emvowel link at the side of the page. A more advanced version is in progress."

tool disemvowelment web comment forum troll language

Why Verbs? - Not The User's Fault

2008 Aug 28, 11:15I can't say why this is funny: "First, the really big picture of what Ubiquity is supposed to be all about: It's a step towards a Web where verbs (i.e. functionality, i.e. commands, i.e. services) are first-class citizens. And that's why I'm thinking it should be renamed from Ubiquity to something like "Mozilla Verbs", maybe."

mozilla firefox ubiquity ui via:ethan_t_hein

Feed Folder Deprecated; Use Internet Explorer 8

2008 Mar 7, 7:20

Internet Explorer 8 has made my plugin Feed Folder obselete in functionality and implementation -- which is good!

IE8's Feed Folder feature screen shot. IE7 Feed Folder plugin screen shot.

I made Feed Folder for IE7 because I wanted the Live Bookmarking feature from FireFox. The Feed Folder plugin for IE7 would allow you to display your feeds as virtual folders in your Links Bar. When your feed is updated the virtual folder is updated as well with the new feed items. I use del.icio.us to store all my links so I could add virtual folders of my daily links, my friends blogs links, quick reference links, etc. etc.

My plugin relied on shell folders to implement the virtual folders I described above, but IE8 doesn't support shell folders in the Favorites Bar. But I'm OK with Feed Folder not working in IE8 since there's a much better implementation already there. IE8 does better than my plugin on a number of points: First, there isn't the horrible perf. issue that my plugin had on Vista. Second, when a feed is updated the virtual folder flashes to note the change in status. Third, unread items are bolded and the bolding bubbles up from feeds contained in subfolders. And lastly, the middle click button is supported to open items in a new tab.

Accordingly, I don't plan to work on Feed Folder anymore unless someone comes up with a good reason. Instead I mark Feed Folder deprecated and suggest you use Internet Explorer 8 instead.

To use this feature in IE8 simply drag a feed from your feed list in your Favorites Center onto your Favorites Bar. Or, when viewing a feed, click on the 'Add to Favorites' Star Plus icon thing in the upper left, and select 'Monitor on Favorites Bar'. A .url Internet Shortcut file is produced as usual, but if you open up the .url file you'll see there's some additional info about the feed.

ie8 feed feedfolder plugin technical browser ie rss

Summary of data visualisation utilities measured by functionality - Swivel

2008 Jan 3, 12:30This just in from Swivel: Swivel is the best!

swivel data visualization website

Functional Requirements for Uniform Resource Names (RFC 1737)

2007 Feb 21, 6:30Standard describing the requirements of URNs.

urn standard reference rfc ietf uri

Representing vCard Objects in RDF/XML

2006 Nov 28, 5:15This note specifies a Resource Description Framework (RDF) encoding of the vCard profile defined by RFC 2426 and to provide equivalent functionality to its standard format. The motivation is to enable the common and consistent description of persons (usin

rdf reference semanticweb specification standards w3c xml vcard schema example

Vroom

2004 Sep 10, 12:04I got my driver's license today. Imagine that. I lost all of the points that I could on the parallel parking portion of the test. You know in California they don't test you on parallel parking ability. With my new license my functional impairment level has moved from minor to non-existent. Hoo-raw. Incidentally if you search for the phrase "stealth geriatrics" on google, the first link is to a power point presentation entitled "stealth geriatrics." I just thought that was a funny phrase.