bug page 3 - Dave's Blog

Time/Date Conversion Tool

I built timestamp.exe, a Windows command line tool to convert between computer and human readable date/time formats mostly for working on the first run wizard for IE8. We commonly write out our dates in binary form to the registry and in order to test and debug my work it became useful to be able to determine to what date the binary value of a FILETIME or SYSTEMTIME corresponded or to produce my own binary value of a FILETIME and insert it into the registry.

For instance, to convert to a binary value:

[PS C:\] timestamp -inString 2009/08/28:10:18 -outHexValue -convert filetime
2009/08/28:10:18 as FILETIME: 00 7c c8 d1 c8 27 ca 01

Converting in the other direction, if you don't know what format the bytes are in, just feed them in and timestamp will try all conversions and list only the valid ones:

[PS C:\] timestamp -inHexValue  "40 52 1c 3b"
40 52 1c 3b as FILETIME: 1601-01-01:00:01:39.171
40 52 1c 3b as Unix Time: 2001-06-05:03:30:08.000
40 52 1c 3b as DOS Time: 2009-08-28:10:18:00.000

[PS C:\] timestamp -inHexValue  "40 52 1c 3b" -convert guess
40 52 1c 3b as DOS Time: 2009-08-28:10:18:00.000

When I first wrote this I had a bug in my function that parses the date-time value string in which I could parse 2009-07-02:10:18 just fine, but I wouldn't be able to parse 2009-09-02:10:18 correctly. This was my code:

success = swscanf_s(timeString, L"%hi%*[\\/- ,]%hi%*[\\/- ,]%hi%*[\\/- ,Tt:.]%hi%*[:.]%hi%*[:.]%hi%*[:.]%hi", 
&systemTime->wYear,
&systemTime->wMonth,
&systemTime->wDay,
&systemTime->wHour,
&systemTime->wMinute,
&systemTime->wSecond,
&systemTime->wMilliseconds) > 1;

To convert between these various forms yourself read The Old New Thing date conversion article or Josh Poley's date time article. I previously wrote about date formats I like and dislike.

Creating Accelerators for Other People's Web Services

Before we shipped IE8 there were no Accelerators, so we had some fun making our own for our favorite web services. I've got a small set of tips for creating Accelerators for other people's web services. I was planning on writing this up as an IE blog post, but Jon wrote a post covering a similar area so rather than write a full and coherent blog post I'll just list a few points:

• The first thing to try is looking for developer help for the web service, specifically if there's a REST-ful URL based API. For example, Bing Maps has great URL API documentation that would be enough to create an Accelerator.
• The Accelerator XML is very similar to HTML forms. If you can find an HTML form for the web service for which you want to create an Accelerator, you can view the HTML source and create an Accelerator based on that.
• I created the FormToAccelerator extension based on the previous idea. You can use the extension to create an Accelerator from an HTML form, or just use it to create the start of one and edit it manually after.
• If the page doesn't use an HTML form, you can start up an HTTP debugger like Fiddler, use the web service from the normal web page, and then in Fiddler see if you can find a REST-ful looking URL you can use.
• When looking to create a preview for your Accelerator, see if the web page for the web service has a mobile version or a version that's intended to embed in other web pages via an iframe. On this same line, iPhone apps make great Accelerators usually with lovely previews.
• If there's no mobile or embeddable version and the only thing wrong with the normal web page for the web service is that the useful information doesn't fit in the preview window then see if you can find an HTML tag with a name or id near the useful information, and stick a '#' fragment pointing to that tag onto the preview URL template.
• Without a reasonable REST-ful API you can use a combination of Google's "site:" and "I'm Feeling Lucky" to find the most relevant page on a particular site.
• The value of a name and value pair need not consist of only a single Accelerator variable. You can get creative and put other text in there. For instance, I implemented a Google currency conversion by setting the query to "{selection} in US Dollars".

Ladybug

sequelguy posted a photo:

The 'Is It UTF-8?' Quick and Dirty Test

I've found while debugging networking in IE its often useful to quickly tell if a string is encoded in UTF-8. You can check for the Byte Order Mark (EF BB BF in UTF-8) but, I rarely see the BOM on UTF-8 strings. Instead I apply a quick and dirty UTF-8 test that takes advantage of the well-formed UTF-8 restrictions.

Unlike other multibyte character encoding forms (see Windows supported character sets or IANA's list of character sets), for example Big5, where sticking together any two bytes is more likely than not to give a valid byte sequence, UTF-8 is more restrictive. And unlike other multibyte character encodings, UTF-8 bytes may be taken out of context and one can still know that its a single byte character, the starting byte of a three byte sequence, etc.

The full rules for well-formed UTF-8 are a little too complicated for me to commit to memory. Instead I've got my own simpler (this is the quick part) set of rules that will be mostly correct (this is the dirty part). For as many bytes in the string as you care to examine, check the most significant digit of the byte:

F:

This is byte 1 of a 4 byte encoded codepoint and must be followed by 3 trail bytes.

E:

This is byte 1 of a 3 byte encoded codepoint and must be followed by 2 trail bytes.

C..D:

This is byte 1 of a 2 byte encoded codepoint and must be followed by 1 trail byte.

8..B:

This is a trail byte.

0..7:

This is a single byte encoded codepoint.

Tab Expansion in PowerShell

PowerShell gives us a real CLI for Windows based around .Net stuff. I don't like the creation of a new shell language but I suppose it makes sense given that they want something C# like but not C# exactly since that's much to verbose and strict for a CLI. One of the functions you can override is the TabExpansion function which is used when you tab complete commands. I really like this and so I've added on to the standard implementation to support replacing a variable name with its value, tab completion of available commands, previous command history, and drive names (there not restricted to just one letter in PS).

Learning the new language was a bit of a chore but MSDN helped. A couple of things to note, a statement that has a return value that you don't do anything with is implicitly the return value for the current function. That's why there's no explicit return's in my TabExpansion function. Also, if you're TabExpansion function fails or returns nothing then the builtin TabExpansion function runs which does just filenames. This is why you can see that the standard TabExpansion function doesn't handle normal filenames: it does extra stuff (like method and property completion on variables that represent .Net objects) but if there's no fancy extra stuff to be done it lets the builtin one take a crack.

Here's my TabExpansion function. Probably has bugs, so watch out!

function EscapePath([string] $path, [string] $original)
{
    if ($path.Contains(' ') -and !$original.Contains(' '))
    {
        '"'   $path   '"';
    }
    else
    {
        $path;
    }
}

function PathRelativeTo($pathDest, $pathCurrent)
{
    if ($pathDest.PSParentPath.ToString().EndsWith($pathCurrent.Path))
    {
        '.\'   $pathDest.name;
    }
    else
    {
        $pathDest.FullName;
    }
}

#  This is the default function to use for tab expansion. It handles simple
# member expansion on variables, variable name expansion and parameter completion
# on commands. It doesn't understand strings so strings containing ; | ( or { may
# cause expansion to fail.

function TabExpansion($line, $lastWord)
{
    switch -regex ($lastWord)
    {
         # Handle property and method expansion...
         '(^.*)(\$(\w|\.) )\.(\w*)$' {
             $method = [Management.Automation.PSMemberTypes] `
                 'Method,CodeMethod,ScriptMethod,ParameterizedProperty'
             $base = $matches[1]
             $expression = $matches[2]
             Invoke-Expression ('$val='   $expression)
             $pat = $matches[4]   '*'
             Get-Member -inputobject $val $pat | sort membertype,name |
                 where { $_.name -notmatch '^[gs]et_'} |
                 foreach {
                     if ($_.MemberType -band $method)
                     {
                         # Return a method...
                         $base   $expression   '.'   $_.name   '('
                     }
                     else {
                         # Return a property...
                         $base   $expression   '.'   $_.name
                     }
                 }
             break;
          }

         # Handle variable name expansion...
         '(^.*\$)([\w\:]*)$' {
             $prefix = $matches[1]
             $varName = $matches[2]
             foreach ($v in Get-Childitem ('variable:'   $varName   '*'))
             {
                 if ($v.name -eq $varName)
                 {
                     $v.value
                 }
                 else
                 {
                    $prefix   $v.name
                 }
             }
             break;
         }

         # Do completion on parameters...
         '^-([\w0-9]*)' {
             $pat = $matches[1]   '*'

             # extract the command name from the string
             # first split the string into statements and pipeline elements
             # This doesn't handle strings however.
             $cmdlet = [regex]::Split($line, '[|;]')[-1]

             #  Extract the trailing unclosed block e.g. ls | foreach { cp
             if ($cmdlet -match '\{([^\{\}]*)$')
             {
                 $cmdlet = $matches[1]
             }

             # Extract the longest unclosed parenthetical expression...
             if ($cmdlet -match '\(([^()]*)$')
             {
                 $cmdlet = $matches[1]
             }

             # take the first space separated token of the remaining string
             # as the command to look up. Trim any leading or trailing spaces
             # so you don't get leading empty elements.
             $cmdlet = $cmdlet.Trim().Split()[0]

             # now get the info object for it...
             $cmdlet = @(Get-Command -type 'cmdlet,alias' $cmdlet)[0]

             # loop resolving aliases...
             while ($cmdlet.CommandType -eq 'alias') {
                 $cmdlet = @(Get-Command -type 'cmdlet,alias' $cmdlet.Definition)[0]
             }

             # expand the parameter sets and emit the matching elements
             foreach ($n in $cmdlet.ParameterSets | Select-Object -expand parameters)
             {
                 $n = $n.name
                 if ($n -like $pat) { '-'   $n }
             }
             break;
         }

         default {
             $varNameStar = $lastWord   '*';

             foreach ($n in @(Get-Childitem $varNameStar))
             {
                 $name = PathRelativeTo ($n) ($PWD);

                 if ($n.PSIsContainer)
                 {
                     EscapePath ($name   '\') ($lastWord);
                 }
                 else
                 {
                     EscapePath ($name) ($lastWord);
                 }
             }

             if (!$varNameStar.Contains('\'))
             {
                foreach ($n in @(Get-Command $varNameStar))
                {
                    if ($n.CommandType.ToString().Equals('Application'))
                    {
                       foreach ($ext in @((cat Env:PathExt).Split(';')))
                       {
                          if ($n.Path.ToString().ToLower().EndsWith(($ext).ToString().ToLower()))
                          {
                              EscapePath($n.Path) ($lastWord);
                          }
                       }
                    }
                    else
                    {
                        EscapePath($n.Name) ($lastWord);
                    }
                }

                foreach ($n in @(Get-psdrive $varNameStar))
                {
                    EscapePath($n.name   ":") ($lastWord);
                }
             }

             foreach ($n in @(Get-History))
             {
                 if ($n.CommandLine.StartsWith($line) -and $n.CommandLine -ne $line)
                 {
                     $lastWord   $n.CommandLine.Substring($line.Length);
                 }
             }

             # Add the original string to the end of the expansion list.
             $lastWord;

             break;
         }
    }
}

QuickBase Formula Pretty Printer and Syntax Highlighter

Sarah asked me if I knew of a syntax highlighter for the QuickBase formula language which she uses at work. I couldn't find one but thought it might be fun to make a QuickBase Formula syntax highlighter based on the QuickBase help's description of the formula syntax. Thankfully the language is relatively simple since my skills with ANTLR, the parser generator, are rusty now and I've only used it previously for personal projects (like Javaish, the ridiculous Java based shell idea I had).

With the help of some great ANTLR examples and an ANTLR cheat sheet I was able to come up with the grammar that parses the QuickBase Formula syntax and prints out the same formula marked up with HTML SPAN tags and various CSS classes. ANTLR produces the parser in Java which I wrapped up in an applet, put in a jar, and embedded in an HTML page. The script in that page runs user input through the applet's parser and sticks the output at the bottom of the page with appropriate CSS rules to highlight and print the formula in a pretty fashion.

What I learned:

• I didn't realize that Java applets are easy to use via script in an HTML page. In the JavaScript I can simply refer to publicly exposed methods on the applet and run JavaScript strings through them. It makes for a great combination: do the heavy coding in Java and do the UI in HTML. I may end up doing this again in the future.
• I love ANTLRWorks, the ANTLR IDE, that didn't exist the last time I used ANTLR. It tells you about issues with your grammar as you create it, lets you easily debug the grammar running it forwards and backwards, display parse trees, and other useful things.

Anecdotes from Work

The names in the following anecdote have been changed. Except for my name (I'm Dave).

I got a new laptop a while back. I had it in my office and Tim came in to ask me something but paused when he saw my laptop. "Oh, is this one of those new touch screen laptops?" he asked, the whole time moving his hand towards my laptop and punctuating his sentence by pressing his finger to the screen. "No" I responded.

Walking down a hallway I heard Winston, one of our managers, say, "Hey Tim!" Winston catches up to me and asks, "Are you almost done with the XYZ bug?" I realized Winston was talking to me and got my name wrong but I figured I'll ignore it and perhaps he'll realize his mistake. Winston continued "I just talked with some people who say they're blocked and waiting for Tim to finish the XYZ bug." "Dave" I said helpfully attempting to diplomatically correct Winston since he apparently hadn't realized his error. "No, it was Jeremy and Bill." Winston said naming the people he had talked to who were waiting for me to fix the XYZ bug. At this point I decided it would be easier to just answer his question and end the conversation than to get into this whole thing. As far as I know, Winston has not gotten my name wrong at any other time.