character page 3 - Dave's Blog

Platonic Ideals in Anathem and The Atrocity Archives

On the topic of made up words: I get made up words for made up things, but there's already a name for cell-phone in English: its "cell-phone". The narrator notes that the book has been translated into English so I guess I'll blame the fictional translator. Anyway, I wasn't bothered by the made up words nearly as much as some folk. Its a good thing I'm long out of college because I can easily imagine confusing the names of actual concepts and people with those from the book, like Hemn space for Hamming distance. Towards the beginning, the description of slines and the post-post-apocalyptic setting reminded me briefly of Idiocracy.

Recently, I've been reading everything of Charles Stross that I can, including about a month ago, The Jennifer Morgue from the surprisingly awesome amalgamation genre of spy thriller and Lovecraft horror. Its the second in a series set in a universe in which magic exists as a form of mathematics and follows Bob Howard programmer/hacker, cube dweller, and begrudging spy who works for a government agency tasked to suppress this knowledge and protect the world from its use. For a taste, try a short story from the series that's freely available on Tor's website, Down on the Farm.

Coincidentally, both Anathem and the Bob Howard series take an interest in the world of Platonic ideals. In the case of Anathem (without spoiling anything) the universe of Platonic ideals, under a different name of course, is debated by the characters to be either just a concept or an actual separate universe and later becomes the underpinning of major events in the book. In the Bob Howard series, magic is applied mathematics that through particular proofs or computations awakens/disturbs/provokes unnamed horrors in the universe of Platonic ideals to produce some desired effect in Bob's universe.

The 'Is It UTF-8?' Quick and Dirty Test

I've found while debugging networking in IE its often useful to quickly tell if a string is encoded in UTF-8. You can check for the Byte Order Mark (EF BB BF in UTF-8) but, I rarely see the BOM on UTF-8 strings. Instead I apply a quick and dirty UTF-8 test that takes advantage of the well-formed UTF-8 restrictions.

Unlike other multibyte character encoding forms (see Windows supported character sets or IANA's list of character sets), for example Big5, where sticking together any two bytes is more likely than not to give a valid byte sequence, UTF-8 is more restrictive. And unlike other multibyte character encodings, UTF-8 bytes may be taken out of context and one can still know that its a single byte character, the starting byte of a three byte sequence, etc.

The full rules for well-formed UTF-8 are a little too complicated for me to commit to memory. Instead I've got my own simpler (this is the quick part) set of rules that will be mostly correct (this is the dirty part). For as many bytes in the string as you care to examine, check the most significant digit of the byte:

F:

This is byte 1 of a 4 byte encoded codepoint and must be followed by 3 trail bytes.

E:

This is byte 1 of a 3 byte encoded codepoint and must be followed by 2 trail bytes.

C..D:

This is byte 1 of a 2 byte encoded codepoint and must be followed by 1 trail byte.

8..B:

This is a trail byte.

0..7:

This is a single byte encoded codepoint.

Subst Allows Non-Letter Drive Letters

I knew that the command line tool subst would create virtual drives that map to existing directories but I didn't know that subst lets you name the virtual drives with characters that aren't US-ASCII letters. For instance you can run 'subst 4: C:\windows' and then 'more 4:\win.ini' to dump C:\windows\win.ini. This also works for non-US-ASCII characters like, "Ｃ" (aka U+FF23, Fullwidth Latin Capital Letter C), which when displayed by cmd.exe via some best fit style character conversions looks just like the regular US-ASCII 'C'. None of Explorer, IE, or the common file dialogs allow the use of these odd virtual drives -- just cmd.exe, so I'm not sure how this would ever be useful but I thought it was odd and I wanted to share.

Party Movies Recommended by Netflix

Netflix has recommended three party movies over my time with Netflix and if you're OK with movies featuring sex, drugs, rock&roll (or techno) as almost the main character then I can recommend at least The Boys and Girls Guide to Getting Down.

24 Hour Party People is based on the true story of Tony Wilson, journalist, band manager, and club owner (not all at once) around the rise of punk and new wave in England. Like many true-story based movies it starts off strong and very interesting but gets very slow at the end like the writers got bored and just started copying the actual events. Unless you have some interest in the history of music in the 80s in Manchester I don't recommend this movie.

Human Traffic is fun and funny following a group of friends going out for a night of clubbing and partying. I had to get over seeing John Simm as not The Master from Doctor Who but rather as a partying youth. It felt like it was geared towards viewers who were on something like the totally odd techno musical interludes with the characters dancing for no apparent reason. Otherwise the movie was good.

The Boys and Girls Guide to Getting Down is done in the style of an old educational movie on the topic of clubbing and partying. It sounds like a premise that would get old but they do a good job. While demonstrating drinking and driving they have scientists push a mouse around in a toy convertible. Enough said. It was funny and I recommend it.

Braid Recommendation

I recently finished Braid, the Xbox Live game, and a comparison with Portal is helpful. From a screen shot Braid looks like a normal 2D platformer, but that's like looking at a screen shot of Portal and saying its a first person shooter. While the scaffolding of the game-play may sort of fall into that category, the games are actually about exploring the character's ability and solving puzzles. In Portal the ability is bending space and in Braid its bending time. However, whereas in Portal there is one space bending mechanism, the portal gun, Braid's protagonist explores several different time bending techniques including, most prominently, reversing time, but also time dilation, multiple time-lines, and other odd things.

Similar to the difference in game-play, while Portal has a strict simplicity to its visual style, Braid is much more ornate, like you're playing in an oil painting. Without seeing video of the game, or playing the demo (which is available for free on Xbox Live) its difficult to convey, but it is quite lovely and the animation adds quite a bit. Both games too are rather short leaving you just a bit hungry for more and have an interesting plot and an ending that I'd hate to spoil although Braid replaces Portal's humor with melancholy. If you enjoyed Portal and Twelve Monkeys then I'd recommend Braid.

Kids in the Hall Live in Seattle

Sarah and I saw the Kids in the Hall "Live As We'll Ever Be" Tour in the WaMu theater in Seattle this past Thursday. I'd only ever seen their television show so it was cool to see them live. I thought that them being in a live format on stage would make the show significantly different, but other than having a bad seat and not being able to see very well, and the Kids sometimes ad-libbing or breaking character, it was like watching their show. It consisted of mostly new material with some returning characters like the Chicken Lady, Buddy Cole, the head crusher, etc. Their Facebook page has two videos that they played during the show.

I've been using the best Kids in the Hall fansite with an archive of searchable transcripts since high school. But now days what with all the new fangled video websites I can link right to some of my favorite sketches from the show. Like the Inexperienced Cannibal.

And the meta-sketch The Raise.

Finally finished Baroque Cycle Novels

I've finally finished the Baroque Cycle, a historical fiction series set in the 17th and 18th centuries by Neal Stephenson whose work I always enjoy. There were often delays where I'd forget about the books until I had to take plane somewhere, or get discouraged reading about the character's thoughts on economics, or have difficulty finding the next volume, or become more engrossed in other books, projects or video games, and leave the Baroque Cycle books untouched for many months at a time. Consequently, my reading of this series has, I'm ashamed to say, spanned years. After finishing some books which I enjoy I end up hungry for just a bit more to read. For this series I don't need a bit more to read, I'm done with that, but I do want a badge or maybe a medal. Or barring that, college credit in European History and Macro Economics. I can recommend this book to anyone who has enjoyed Neal Stephenson's other work and has a few years of free time to kill.

URI Fragment Info Roundup

Information about URI Fragments, the portion of URIs that follow the '#' at the end and that are used to navigate within a document, is scattered throughout various documents which I usually have to hunt down. Instead I'll link to them all here.

Definitions. Fragments are defined in the URI RFC which states that they're used to identify a secondary resource that is related to the primary resource identified by the URI as a subset of the primary, a view of the primary, or some other resource described by the primary. The interpretation of a fragment is based on the mime type of the primary resource. Tim Berners-Lee notes that determining fragment meaning from mime type is a problem because a single URI may contain a single fragment, however over HTTP a single URI can result in the same logical resource represented in different mime types. So there's one fragment but multiple mime types and so multiple interpretations of the one fragment. The URI RFC says that if an author has a single resource available in multiple mime types then the author must ensure that the various representations of a single resource must all resolve fragments to the same logical secondary resource. Depending on which mime types you're dealing with this is either not easy or not possible.

HTTP. In HTTP when URIs are used, the fragment is not included. The General Syntax section of the HTTP standard says it uses the definitions of 'URI-reference' (which includes the fragment), 'absoluteURI', and 'relativeURI' (which don't include the fragment) from the URI RFC. However, the 'URI-reference' term doesn't actually appear in the BNF for the protocol. Accordingly the headers like 'Request-URI', 'Content-Location', 'Location', and 'Referer' which include URIs are defined with 'absoluteURI' or 'relativeURI' and don't include the fragment. This is in keeping with the original fragment definition which says that the fragment is used as a view of the original resource and consequently only needed for resolution on the client. Additionally, the URI RFC explicitly notes that not including the fragment is a privacy feature such that page authors won't be able to stop clients from viewing whatever fragments the client chooses. This seems like an odd claim given that if the author wanted to selectively restrict access to portions of documents there are other options for them like breaking out the parts of a single resource to which the author wishes to restrict access into separate resources.

HTML. In HTML, the HTML mime type RFC defines HTML's fragment use which consists of fragments referring to elements with a corresponding 'id' attribute or one of a particular set of elements with a corresponding 'name' attribute. The HTML spec discusses fragment use additionally noting that the names and ids must be unique in the document and that they must consist of only US-ASCII characters. The ID and NAME attributes are further restricted in section 6 to only consist of alphanumerics, the hyphen, period, colon, and underscore. This is a subset of the characters allowed in the URI fragment so no encoding is discussed since technically its not needed. However, practically speaking, browsers like FireFox and Internet Explorer allow for names and ids containing characters outside of the defined set including characters that must be percent-encoded to appear in a URI fragment. The interpretation of percent-encoded characters in fragments for HTML documents is not consistent across browsers (or in some cases within the same browser) especially for the percent-encoded percent.

Text. Text/plain recently got a fragment definition that allows fragments to refer to particular lines or characters within a text document. The scheme no longer includes regular expressions, which disappointed me at first, but in retrospect is probably good idea for increasing the adoption of this fragment scheme and for avoiding the potential for ubiquitous DoS via regex. One of the authors also notes this on his blog. I look forward to the day when this scheme is widely implemented.

XML. XML has the XPointer framework to define its fragment structure as noted by the XML mime type definition. XPointer consists of a general scheme that contains subschemes that identify a subset of an XML document. Its too bad such a thing wasn't adopted for URI fragments in general to solve the problem of a single resource with multiple mime type representations. I wrote more about XPointer when I worked on hacking XPointer into IE.

SVG and MPEG. Through the Media Fragments Working Group I found a couple more fragment scheme definitions. SVG's fragment scheme is defined in the SVG documentation and looks similar to XML's. MPEG has one defined but I could only find it as an ISO document "Text of ISO/IEC FCD 21000-17 MPEG-12 FID" and not as an RFC which is a little disturbing.

AJAX. AJAX websites have used fragments as an escape hatch for two issues that I've seen. The first is getting a unique URL for versions of a page that are produced on the client by script. The fragment may be changed by script without forcing the page to reload. This goes outside the rules of the standards by using HTML fragments in a fashion not called out by the HTML spec. but it does seem to be inline with the spirit of the fragment in that it is a subview of the original resource and interpretted client side. The other hack-ier use of the fragment in AJAX is for cross domain communication. The basic idea is that different frames or windows may not communicate in normal fashions if they have different domains but they can view each other's URLs and accordingly can change their own fragments in order to send a message out to those who know where to look. IMO this is not inline with the spirit of the fragment but is rather a cool hack.

Encoding methods in C#

For Encode-O-Matic, my encoding tool written in C#, I had to figure out the appropriate DllImport declarations to use IDN Win32 functions which was a pain. To spare others that pain here's the two files CharacterSetEncoding.cs and NationalLanguageSupportUtilities.cs that declare the DllImports for IdnToUnicode, IdnToAscii, NormalizeString, MultiByteToWideChar, and WideCharToMultiByte.