pub page 2 - Dave's Blog

Search
My timeline on Mastodon

URI functions in Windows Store Applications

2013 Jul 25, 1:00

Summary

The Modern SDK contains some URI related functionality as do libraries available in particular projection languages. Unfortunately, collectively these APIs do not cover all scenarios in all languages. Specifically, JavaScript and C++ have no URI building APIs, and C++ additionally has no percent-encoding/decoding APIs.
WinRT (JS and C++)
JS Only
C++ Only
.NET Only
Parse
 
Build
Normalize
Equality
 
 
Relative resolution
Encode data for including in URI property
Decode data extracted from URI property
Build Query
Parse Query
The Windows.Foudnation.Uri type is not projected into .NET modern applications. Instead those applications use System.Uri and the platform ensures that it is correctly converted back and forth between Windows.Foundation.Uri as appropriate. Accordingly the column marked WinRT above is applicable to JS and C++ modern applications but not .NET modern applications. The only entries above applicable to .NET are the .NET Only column and the WwwFormUrlDecoder in the bottom left which is available to .NET.

Scenarios

Parse

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS, and by System.Uri in .NET.
Parsing a URI pulls it apart into its basic components without decoding or otherwise modifying the contents.
var uri = new Windows.Foundation.Uri("http://example.com/path%20segment1/path%20segment2?key1=value1&key2=value2");
console.log(uri.path);// /path%20segment1/path%20segment2

WsDecodeUrl (C++)

WsDecodeUrl is not suitable for general purpose URI parsing.  Use Windows.Foundation.Uri instead.

Build (C#)

URI building is only available in C# via System.UriBuilder.
URI building is the inverse of URI parsing: URI building allows the developer to specify the value of basic components of a URI and the API assembles them into a URI. 
To work around the lack of a URI building API developers will likely concatenate strings to form their URIs.  This can lead to injection bugs if they don’t validate or encode their input properly, but if based on trusted or known input is unlikely to have issues.
            Uri originalUri = new Uri("http://example.com/path1/?query");
            UriBuilder uriBuilder = new UriBuilder(originalUri);
            uriBuilder.Path = "/path2/";
            Uri newUri = uriBuilder.Uri; // http://example.com/path2/?query

WsEncodeUrl (C++)

WsEncodeUrl, in addition to building a URI from components also does some encoding.  It encodes non-US-ASCII characters as UTF8, the percent, and a subset of gen-delims based on the URI property: all :/?#[]@ are percent-encoded except :/@ in the path and :/?@ in query and fragment.
Accordingly, WsEncodeUrl is not suitable for general purpose URI building.  It is acceptable to use in the following cases:
- You’re building a URI out of non-encoded URI properties and don’t care about the difference between encoded and decoded characters.  For instance you’re the only one consuming the URI and you uniformly decode URI properties when consuming – for instance using WsDecodeUrl to consume the URI.
- You’re building a URI with URI properties that don’t contain any of the characters that WsEncodeUrl encodes.

Normalize

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET.  Normalization is applied during construction of the Uri object.
URI normalization is the application of URI normalization rules (including DNS normalization, IDN normalization, percent-encoding normalization, etc.) to the input URI.
        var normalizedUri = new Windows.Foundation.Uri("HTTP://EXAMPLE.COM/p%61th foo/");
        console.log(normalizedUri.absoluteUri); // http://example.com/path%20foo/
This is modulo Win8 812823 in which the Windows.Foundation.Uri.AbsoluteUri property returns a normalized IRI not a normalized URI.  This bug does not affect System.Uri.AbsoluteUri which returns a normalized URI.

Equality

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET. 
URI equality determines if two URIs are equal or not necessarily equal.
            var uri1 = new Windows.Foundation.Uri("HTTP://EXAMPLE.COM/p%61th foo/"),
                uri2 = new Windows.Foundation.Uri("http://example.com/path%20foo/");
            console.log(uri1.equals(uri2)); // true

Relative resolution

This functionality is provided by the WinRT API Windows.Foundation.Uri in C++ and JS and by System.Uri in .NET 
Relative resolution is a function that given an absolute URI A and a relative URI B, produces a new absolute URI C.  C is the combination of A and B in which the basic components specified in B override or combine with those in A under rules specified in RFC 3986.
        var baseUri = new Windows.Foundation.Uri("http://example.com/index.html"),
            relativeUri = "/path?query#fragment",
            absoluteUri = baseUri.combineUri(relativeUri);
        console.log(baseUri.absoluteUri);       // http://example.com/index.html
        console.log(absoluteUri.absoluteUri);   // http://example.com/path?query#fragment

Encode data for including in URI property

This functionality is available in JavaScript via encodeURIComponent and in C# via System.Uri.EscapeDataString. Although the two methods mentioned above will suffice for this purpose, they do not perform exactly the same operation.
Additionally we now have Windows.Foundation.Uri.EscapeComponent in WinRT, which is available in JavaScript and C++ (not C# since it doesn’t have access to Windows.Foundation.Uri).  This is also slightly different from the previously mentioned mechanisms but works best for this purpose.
Encoding data for inclusion in a URI property is necessary when constructing a URI from data.  In all the above cases the developer is dealing with a URI or substrings of a URI and so the strings are all encoded as appropriate. For instance, in the parsing example the path contains “path%20segment1” and not “path segment1”.  To construct a URI one must first construct the basic components of the URI which involves encoding the data.  For example, if one wanted to include “path segment / example” in the path of a URI, one must percent-encode the ‘ ‘ since it is not allowed in a URI, as well as the ‘/’ since although it is allowed, it is a delimiter and won’t be interpreted as data unless encoded.
If a developer does not have this API provided they can write it themselves.  Percent-encoding methods appear simple to write, but the difficult part is getting the set of characters to encode correct, as well as handling non-US-ASCII characters.
        var uri = new Windows.Foundation.Uri("http://example.com" +
            "/" + Windows.Foundation.Uri.escapeComponent("path segment / example") +
            "?key=" + Windows.Foundation.Uri.escapeComponent("=&?#"));
        console.log(uri.absoluteUri); // http://example.com/path%20segment%20%2F%20example?key=%3D%26%3F%23

WsEncodeUrl (C++)

In addition to building a URI from components, WsEncodeUrl also percent-encodes some characters.  However the API is not recommend for this scenario given the particular set of characters that are encoded and the convoluted nature in which a developer would have to use this API in order to use it for this purpose.
There are no general purpose scenarios for which the characters WsEncodeUrl encodes make sense: encode the %, encode a subset of gen-delims but not also encode the sub-delims.  For instance this could not replace encodeURIComponent in a C++ version of the following code snippet since if ‘value’ contained ‘&’ or ‘=’ (both sub-delims) they wouldn’t be encoded and would be confused for delimiters in the name value pairs in the query:
"http://example.com/?key=" + Windows.Foundation.Uri.escapeComponent(value)
Since WsEncodeUrl produces a string URI, to obtain the property they want to encode they’d need to parse the resulting URI.  WsDecodeUrl won’t work because it decodes the property but Windows.Foundation.Uri doesn’t decode.  Accordingly the developer could run their string through WsEncodeUrl then Windows.Foundation.Uri to extract the property.

Decode data extracted from URI property

This functionality is available in JavaScript via decodeURIComponent and in C# via System.Uri.UnescapeDataString. Although the two methods mentioned above will suffice for this purpose, they do not perform exactly the same operation.
Additionally we now also have Windows.Foundation.Uri.UnescapeComponent in WinRT, which is available in JavaScript and C++ (not C# since it doesn’t have access to Windows.Foundation.Uri).  This is also slightly different from the previously mentioned mechanisms but works best for this purpose.
Decoding is necessary when extracting data from a parsed URI property.  For example, if a URI query contains a series of name and value pairs delimited by ‘=’ between names and values, and by ‘&’ between pairs, one must first parse the query into name and value entries and then decode the values.  It is necessary to make this an extra step separate from parsing the URI property so that sub-delimiters (in this case ‘&’ and ‘=’) that are encoded will be interpreted as data, and those that are decoded will be interpreted as delimiters.
If a developer does not have this API provided they can write it themselves.  Percent-decoding methods appear simple to write, but have some tricky parts including correctly handling non-US-ASCII, and remembering not to decode .
In the following example, note that if unescapeComponent were called first, the encoded ‘&’ and ‘=’ would be decoded and interfere with the parsing of the name value pairs in the query.
            var uri = new Windows.Foundation.Uri("http://example.com/?foo=bar&array=%5B%27%E3%84%93%27%2C%27%26%27%2C%27%3D%27%2C%27%23%27%5D");
            uri.query.substr(1).split("&").forEach(
                function (keyValueString) {
                    var keyValue = keyValueString.split("=");
                    console.log(Windows.Foundation.Uri.unescapeComponent(keyValue[0]) + ": " + Windows.Foundation.Uri.unescapeComponent(keyValue[1]));
                    // foo: bar
                    // array: ['','&','=','#']
                });

WsDecodeUrl (C++)

Since WsDecodeUrl decodes all percent-encoded octets it could be used for general purpose percent-decoding but it takes a URI so would require the dev to construct a stub URI around the string they want to decode.  For example they could prefix “http:///#” to their string, run it through WsDecodeUrl and then extract the fragment property.  It is convoluted but will work correctly.

Parse Query

The query of a URI is often encoded as application/x-www-form-urlencoded which is percent-encoded name value pairs delimited by ‘&’ between pairs and ‘=’ between corresponding names and values.
In WinRT we have a class to parse this form of encoding using Windows.Foundation.WwwFormUrlDecoder.  The queryParsed property on the Windows.Foundation.Uri class is of this type and created with the query of its Uri:
    var uri = Windows.Foundation.Uri("http://example.com/?foo=bar&array=%5B%27%E3%84%93%27%2C%27%26%27%2C%27%3D%27%2C%27%23%27%5D");
    uri.queryParsed.forEach(
        function (pair) {
            console.log("name: " + pair.name + ", value: " + pair.value);
            // name: foo, value: bar
            // name: array, value: ['','&','=','#']
        });
    console.log(uri.queryParsed.getFirstValueByName("array")); // ['','&','=','#']
The QueryParsed property is only on Windows.Foundation.Uri and not System.Uri and accordingly is not available in .NET.  However the Windows.Foundation.WwwFormUrlDecoder class is available in C# and can be used manually:
            Uri uri = new Uri("http://example.com/?foo=bar&array=%5B%27%E3%84%93%27%2C%27%26%27%2C%27%3D%27%2C%27%23%27%5D");
            WwwFormUrlDecoder decoder = new WwwFormUrlDecoder(uri.Query);
            foreach (IWwwFormUrlDecoderEntry entry in decoder)
            {
                System.Diagnostics.Debug.WriteLine("name: " + entry.Name + ", value: " + entry.Value);
                // name: foo, value: bar
                // name: array, value: ['','&','=','#']
            }
 

Build Query

To build a query of name value pairs encoded as application/x-www-form-urlencoded there is no WinRT API to do this directly.  Instead a developer must do this manually making use of the code described in “Encode data for including in URI property”.
In terms of public releases, this property is only in the RC and later builds.
For example in JavaScript a developer may write:
            var uri = new Windows.Foundation.Uri("http://example.com/"),
                query = "?" + Windows.Foundation.Uri.escapeComponent("array") + "=" + Windows.Foundation.Uri.escapeComponent("['','&','=','#']");
 
            console.log(uri.combine(new Windows.Foundation.Uri(query)).absoluteUri); // http://example.com/?array=%5B'%E3%84%93'%2C'%26'%2C'%3D'%2C'%23'%5D
 
PermalinkCommentsc# c++ javascript technical uri windows windows-runtime windows-store

C++ constructor member initializers run in member declaration order

2013 Jul 18, 3:29

TL;DR: Keep your C++ class member declaration order the same as your constructor member initializers order.

C++ guarantees that the member initializers in a constructor are called in order. However the order in which they are called is the order in which the associated members are declared in the class, not the order in which they appear in the member initializer list. For instance, take the following code. I would have thought it would print "three, one, two", but in fact it prints, "one, two, three".

   
#include "stdafx.h"
#include

class PrintSomething {
public:
PrintSomething(const wchar_t *name) { std::wcout << name << std::endl; }
};

class NoteOrder {
public:
// This order doesn't matter.
NoteOrder() : three(L"three"), one(L"one"), two(L"two") { }

PrintSomething one;
PrintSomething two;
PrintSomething three;
};

int wmain(const int argc, const wchar_t* argv[])
{
NoteOrder note; // Prints one, two, three, not three, one, two!
return 0;
}
PermalinkCommentsc++ development programming technical

Windows Store on Windows 8 Fun For Independent Developers

2013 Jun 24, 1:00
Having worked on Windows 8 I'm not in a neutral position to review aspects of it, however I'll say from a high level I love taking the following various positives from smart phone apps and app stores and applying it to the desktop:
  • Independent developers can easily publish apps.
  • One trusted place for a user to find apps.
  • User can trust apps are limited to a declared set of capabilities.
  • One common and easy way for users to buy and try apps.
  • Easy mechanism for independent developers to collect revenue.
Relieving the independent developer of software development overhead, in this case Windows taking care of distribution and sales infrastructure is wonderful for me with my third party developer hat on. This combined with my new found fun of developing in JavaScript and the new Windows Runtime APIs means I've been implementing and finishing various ideas I've had - some for fun and some for productivity on my Surface. Development notes to follow.
PermalinkCommentsstore technical windows windows-store

Ben Goldacre’s TED talk on publication bias, drug...

2012 Sep 28, 3:55


drug companies hiding the results of clinical trials.

(via I did a new talk at TED, on drug companies and hidden data.)

PermalinkCommentsscience video ted

Patent Office tries “Stack Overflow for patents” to find prior art | Ars Technica

2012 Sep 20, 2:27

Welcome news. Glad to hear they’re looking for improvements.

… the USPTO has also worked with Stack Exchange, … to create a new site called Ask Patents. … Examiners or others looking for prior art can post questions about a specific application, and members of the general public can respond with evidence that an applicant was not the first to invent the subject matter of the application.

PermalinkCommentsip law patent stack-exchange technical uspto

AbleGamers creates how-to accessibility guide for devs, publishers | Joystiq

2012 Sep 14, 4:20

A howto on making your video game accessible to those with disabilities (blind, color blind, deaf, etc)

The AbleGamers Foundation has created a 48-page document it hopes will serve as a how-to guide for game developers and publishers on how to create more accessible games. The in-depth guide by the disability non-profit covers in great detail how to make games available to those with varying degrees of mobility, hearing, visual and cognitive issues.

PermalinkCommentsgame technical video-game programming accessibility howto

Newsroom: Miscellaneous: New Online Tool Gives Public Wider Access to Key U.S. Statistics

2012 Jul 28, 2:35

The U.S. Census Bureau today released a new online service that makes key demographic, socio-economic and housing statistics more accessible than ever before. The Census Bureau’s first-ever public Application Programming Interface (API) allows developers to design Web and mobile apps to explore or learn more about America’s changing population and economy.

PermalinkCommentstechnical api census statistics stats web restful rest

wired: cnet: Mp3 playing retainer transmits music through your...

2012 Jul 3, 2:31




wired:

cnet:

Mp3 playing retainer transmits music through your teeth:

Bone conduction audio, retainers, and shiny hip-hop teeth grills aren’t new inventions, but tech hacker Aisen Caro Chacin had the clever idea to put them all together.

The Play-A-Grill MP3 player prototype fits in your mouth like a retainer, shines on the outside like a precious metal rap grill, and plays music through bone conduction through your teeth.

Read more

Oh man, this would have made puberty just a touch cooler. Maybe. 

PermalinkCommentshumor mp3 tech

Mike Merrill Is a Publicly Traded Person | Motherboard

2012 Jul 1, 2:01

By which is meant the following:

Essentially, each shareholder has decided to buy into Merrill ’s life and help him make better choices that will then hopefully up the stock price, allowing them to make money should they ever decide to cash out.

What? Oh, he lives in Portland.

PermalinkCommentsMike-Merrill humor economics stock

ifc: This week on Comedy Bang! Bang! - Michael Cera!

2012 Jun 27, 3:38


ifc:

This week on Comedy Bang! Bang! - Michael Cera!

PermalinkCommentshumor comedy-bang-bang michael-cera video

Crypto breakthrough shows Flame was designed by world-class scientists | Ars Technica

2012 Jun 7, 9:12

So this is another Stuxnet by Israel/US?

The analysis reinforces theories that researchers from Kaspersky Lab, CrySyS Lab, and Symantec published almost two weeks ago. Namely, Flame could only have been developed with the backing of a wealthy nation-state. … “It’s not a garden-variety collision attack, or just an implementation of previous MD5 collisions papers—which would be difficult enough,” Matthew Green, a professor specializing in cryptography in the computer science department at Johns Hopkins University, told Ars. “There were mathematicians doing new science to make Flame work.”

PermalinkCommentstechnical security web internet md5 cryptography flame

Scorching legal response from Dajaz1.com to the unsealed US gov't docs on the illegal, sleazy seizure of its domain name

2012 May 7, 1:25

Its all quite shocking.

Fourth , when I explained that the blog publisher had received music from the industry itself, a government attorney replied that authorization was an “affirmative defense” that need not be taken into account by the government in carrying out the seizure. That was stunning.

PermalinkCommentsmusic law ip riaa dns goverment legal

Another Comedy Bang Bang preview clip this time with Zach...

2012 Apr 18, 6:02


Another Comedy Bang Bang preview clip this time with Zach Galifianakis.

PermalinkCommentszach-galifianakis comedy-bang-bang video humor preview scott-aukerman tv clip

WHATWG Weekly: http+aes URL scheme, control Referer, …

2012 Mar 7, 8:08

Seems generally bad to embed sensitive info in the URI (the http+aes URI scheme’s decryption key) similar to the now deprecated password field.

Use case is covered here: http://lists.w3.org/Archives/Public/ietf-http-wg/2012JanMar/0811.html.  Also discussion including someone mentioning the issue above.

PermalinkCommentstechnical html5 html uri uri-scheme http http+aes

Client Side Cross Domain Data YQL Hack

2012 Feb 27, 2:28

One of the more limiting issues of writing client side script in the browser is the same origin limitations of XMLHttpRequest. The latest version of all browsers support a subset of CORS to allow servers to opt-in particular resources for cross-domain access. Since IE8 there's XDomainRequest and in all other browsers (including IE10) there's XHR L2's cross-origin request features. But the vast majority of resources out on the web do not opt-in using CORS headers and so client side only web apps like a podcast player or a feed reader aren't doable.

One hack-y way around this I've found is to use YQL as a CORS proxy. YQL applies the CORS header to all its responses and among its features it allows a caller to request an arbitrary XML, HTML, or JSON resource. So my network helper script first attempts to access a URI directly using XDomainRequest if that exists and XMLHttpRequest otherwise. If that fails it then tries to use XDR or XHR to access the URI via YQL. I wrap my URIs in the following manner, where type is either "html", "xml", or "json":

        yqlRequest = function(uri, method, type, onComplete, onError) {
var yqlUri = "http://query.yahooapis.com/v1/public/yql?q=" +
encodeURIComponent("SELECT * FROM " + type + ' where url="' + encodeURIComponent(uri) + '"');

if (type == "html") {
yqlUri += encodeURIComponent(" and xpath='/*'");
}
else if (type == "json") {
yqlUri += "&callback=&format=json";
}
...

This also means I can get JSON data itself without having to go through JSONP.
PermalinkCommentsxhr javascript yql client-side technical yahoo xdr cors

Blackmail DRM - Stolen Thoughts

2012 Feb 13, 4:00

Most existing DRM attempts to only allow the user to access the DRM'ed content with particular applications or with particular credentials so that if the file is shared it won't be useful to others. A better solution is to encode any of the user's horrible secrets into unique versions of the DRM'ed content so that the user won't want to share it. Entangle the users and the content provider's secrets together in one document and accordingly their interests. I call this Blackmail DRM. For an implementation it is important to point out that the user's horrible secret doesn't need to be verified as accurate, but merely verified as believable.

Apparently I need to get these blog posts written faster because only recently I read about Social DRM which is a light weight version of my idea but with a misleading name. Instead of horrible secrets, they say they'll use personal information like the user's name in the DRM'ed content. More of my thoughts stolen and before I even had a chance to think of it first!

PermalinkCommentsdrm blackmail blackmail-drm technical humor social-drm

DRM is to publishing as science was to Stalinism

2012 Jan 30, 9:11

I hadn’t heard of “Social DRM” (described in this article). Sounds like my blackmail DRM idea.

PermalinkCommentsdrm publishing

Django Reinhardt-01-10 : Django Reinhardt : Free Download & Streaming : Internet Archive

2012 Jan 22, 4:40

This is Django Reinhardt’s Gypsy swing from the 30s and 40s on archive.org and it is all in the public domain. I didn’t know the term for the genre so it took me a while to find this.

PermalinkCommentsmusic public-domain django-reinhardt jazz gypsy-jazz

Copyright King: Why the "I Have a Dream" Speech Still Isn't Free (vice.com)

2012 Jan 17, 9:37

There’s weird stuff you’d think is public domain but isn’t including Martin Luther King Jr.‘s “I Have a Dream” speech. FTA: ”If you want to watch the whole thing, legally, you’ll need to get the $20 DVD.

That’s because the King estate, and, as of 2009, the British music publishing conglomerate EMI Publishing, owns the copyright of the speech and its recorded performance.”

PermalinkCommentscopyright mlk speech public-domain

James Joyce and Virginia Woolf copyrights expire tonight

2011 Dec 31, 2:35

More amazing works entering public domain. A shame that US works take such an extreme amount of time.

PermalinkCommentslegal public-domain copyright
Older EntriesNewer Entries Creative Commons License Some rights reserved.