index page 2 - Dave's Blog

JavaScript Array methods in the latest browsers

Android eBook Reader And Makers

I was reading Makers, Cory Doctorow's latest novel, as it was serialized on Tor's website but with no ability to save my place within a page I set out to find a book reading app for my G1 Android phone. I stopped looking once I found Aldiko. Its got bookmarks within chapters, configurable fonts, you can look-up words in a dictionary, and has an easy method to download public domain and creative common books. I was able to take advantage of Aldiko's in-app book download system to get Makers onto my phone so I didn't have to bother with any conversion programs etc, and I didn't have to worry about spacing or layout, the book had the correct cover art, and chapter delimiters. I'm very happy with this app and finished reading Makers on it.

Makers is set in the near future and features teams of inventors, networked 3d printers, IP contention, body modifications, and Disney -- just the sort of thing you'd expect from a Cory Doctorow novel. The tale seems to be an allegory for the Internet including displacing existing businesses and the conflict between the existing big entertainment IP owners and the plethora of fans and minor content producers. The story is engaging and the characters filled out and believable. I recommend Makers and as always its Creative Commons so go take a look right now.

PowerShell Scanning Script

I've hooked up the printer/scanner to the Media Center PC since I leave that on all the time anyway so we can have a networked printer. I wanted to hook up the scanner in a somewhat similar fashion but I didn't want to install HP's software (other than the drivers of course). So I've written my own script for scanning in PowerShell that does the following:

1. Scans using the Windows Image Acquisition APIs via COM
2. Runs OCR on the image using Microsoft Office Document Imaging via COM (which may already be on your PC if you have Office installed)
3. Converts the image to JPEG using .NET Image APIs
4. Stores the OCR text into the EXIF comment field using .NET Image APIs (which means Windows Search can index the image by the text in the image)
5. Moves the image to the public share

Here's the actual code from my scan.ps1 file:

param([Switch] $ShowProgress, [switch] $OpenCompletedResult)

$filePathTemplate = "C:\users\public\pictures\scanned\scan {0} {1}.{2}";
$time = get-date -uformat "%Y-%m-%d";

[void]([reflection.assembly]::loadfile( "C:\Windows\Microsoft.NET\Framework\v2.0.50727\System.Drawing.dll"))

$deviceManager = new-object -ComObject WIA.DeviceManager
$device = $deviceManager.DeviceInfos.Item(1).Connect();

foreach ($item in $device.Items) {
        $fileIdx = 0;
        while (test-path ($filePathTemplate -f $time,$fileIdx,"*")) {
                [void](++$fileIdx);
        }

        if ($ShowProgress) { "Scanning..." }

        $image = $item.Transfer();
        $fileName = ($filePathTemplate -f $time,$fileIdx,$image.FileExtension);
        $image.SaveFile($fileName);
        clear-variable image

        if ($ShowProgress) { "Running OCR..." }

        $modiDocument = new-object -comobject modi.document;
        $modiDocument.Create($fileName);
        $modiDocument.OCR();
        if ($modiDocument.Images.Count -gt 0) {
                $ocrText = $modiDocument.Images.Item(0).Layout.Text.ToString().Trim();
                $modiDocument.Close();
                clear-variable modiDocument

                if (!($ocrText.Equals(""))) {
                        $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $fileName
                        if (!($fileName.EndsWith(".jpg") -or $fileName.EndsWith(".jpeg"))) {
                                if ($ShowProgress) { "Converting to JPEG..." }

                                $newFileName = ($filePathTemplate -f $time,$fileIdx,"jpg");
                                $fileAsImage.Save($newFileName, [System.Drawing.Imaging.ImageFormat]::Jpeg);
                                $fileAsImage.Dispose();
                                del $fileName;

                                $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $newFileName 
                                $fileName = $newFileName
                        }

                        if ($ShowProgress) { "Saving OCR Text..." }

                        $property = $fileAsImage.PropertyItems[0];
                        $property.Id = 40092;
                        $property.Type = 1;
                        $property.Value = [system.text.encoding]::Unicode.GetBytes($ocrText);
                        $property.Len = $property.Value.Count;
                        $fileAsImage.SetPropertyItem($property);
                        $fileAsImage.Save(($fileName + ".new"));
                        $fileAsImage.Dispose();
                        del $fileName;
                        ren ($fileName + ".new") $fileName
                }
        }
        else {
                $modiDocument.Close();
                clear-variable modiDocument
        }

        if ($ShowProgress) { "Done." }

        if ($OpenCompletedResult) {
                . $fileName;
        }
        else {
                $result = dir $fileName;
                $result | add-member -membertype noteproperty -name OCRText -value $ocrText
                $result
        }
}

I ran into a few issues:

• MODI doesn't seem to be in the Office 2010 Technical Preview I installed first. Installing Office 2007 fixed that.
• The MODI.Document class, at least via PowerShell, can't be instantiated in a 64bit environment. To run the script on my 64bit OS I had to start powershell from the 32bit cmd.exe (C:\windows\syswow64\cmd.exe).
• I was planning to hook up my script to the scanner's 'Scan' button, but HP didn't get the button working for their Vista driver. Their workaround is "don't do that!".
• You must call Image.Dispose() to get .NET to release its reference to the corresponding image file.
• In trying to figure out how to store the text in the files comment, I ran into a dead-end trying to find the corresponding setter for GetDetailsOf which folks like James O'Neil use in PowerShell for interesting ends.

Platonic Ideals in Anathem and The Atrocity Archives

On the topic of made up words: I get made up words for made up things, but there's already a name for cell-phone in English: its "cell-phone". The narrator notes that the book has been translated into English so I guess I'll blame the fictional translator. Anyway, I wasn't bothered by the made up words nearly as much as some folk. Its a good thing I'm long out of college because I can easily imagine confusing the names of actual concepts and people with those from the book, like Hemn space for Hamming distance. Towards the beginning, the description of slines and the post-post-apocalyptic setting reminded me briefly of Idiocracy.

Recently, I've been reading everything of Charles Stross that I can, including about a month ago, The Jennifer Morgue from the surprisingly awesome amalgamation genre of spy thriller and Lovecraft horror. Its the second in a series set in a universe in which magic exists as a form of mathematics and follows Bob Howard programmer/hacker, cube dweller, and begrudging spy who works for a government agency tasked to suppress this knowledge and protect the world from its use. For a taste, try a short story from the series that's freely available on Tor's website, Down on the Farm.

Coincidentally, both Anathem and the Bob Howard series take an interest in the world of Platonic ideals. In the case of Anathem (without spoiling anything) the universe of Platonic ideals, under a different name of course, is debated by the characters to be either just a concept or an actual separate universe and later becomes the underpinning of major events in the book. In the Bob Howard series, magic is applied mathematics that through particular proofs or computations awakens/disturbs/provokes unnamed horrors in the universe of Platonic ideals to produce some desired effect in Bob's universe.

Subst Allows Non-Letter Drive Letters

I knew that the command line tool subst would create virtual drives that map to existing directories but I didn't know that subst lets you name the virtual drives with characters that aren't US-ASCII letters. For instance you can run 'subst 4: C:\windows' and then 'more 4:\win.ini' to dump C:\windows\win.ini. This also works for non-US-ASCII characters like, "Ｃ" (aka U+FF23, Fullwidth Latin Capital Letter C), which when displayed by cmd.exe via some best fit style character conversions looks just like the regular US-ASCII 'C'. None of Explorer, IE, or the common file dialogs allow the use of these odd virtual drives -- just cmd.exe, so I'm not sure how this would ever be useful but I thought it was odd and I wanted to share.

G1 Android Phone

I finally replaced my old regular cell-phone which was literally being held together by a rubber band with a fancy new G1, my first Internet accessible phone.

I had to call the T-Mobile support line to get data added to my plan and the person helping me was disconcertingly friendly. She asked about my weekend plans and so I felt compelled to ask her the same. Her plans involved replacing her video card so she could get back to World of Warcraft and do I enjoy computer gaming? I couldn't tell if she was genuine or if she was signing me up for magazines.

I was with Sarah in her new car, trying out the phone's GPS functionality via Google Maps while she drove. I switched to Street View and happened to find my car. It was a weird feeling, kind of like those Google conspiracy videos.

The phone runs Google's open source OS and I really enjoy the application API. Its all in Java and URIs and mime-types are sort of basics. Rather than invoking the builtin item picker control directly you invoke an 'intent' specifying the URI of your list of items, a mime-type describing the type of items in the list, and an action 'PICK' and whatever is registered as the picker on the system pops up and lets the user pick from that list. The same goes if you want to 'EDIT' an image, or 'VIEW' an mp3.

I wanted to replace the Google search box gadget that appears on the home screen with my own search box widget that uses OpenSearch descriptors but apparently in the current API you can't make home screen gadgets without changing parts of the OS. My other desired application is something to replace this GPS photo tracker device by recording my location to a file and an additional program on my computer to apply those locations to photos.

Halloween and Gas Park Weekend

The weekend before last Sarah and I went down to Gas Works Park in Seattle. Gas Works Park is a former Seattle Gas Light Company gasification plant now turned into a park with the machinery kept intact and found right on the shore of Lake Union. There's a large hill right next to the plant with an embedded art installation from which you get an excellent view of the park and the lake. Anyway a very cool place. Afer, we ate at Julia's of Wallingford where I stereotypically had the Santa Cruz omelet. Good food, nice place, nice neighborhood.

This past weekend was Halloween weekend. On Halloween at Microsoft parents bring their kids around the office buildings and collect candy from those who have candy in their office. See Matt's photo of one such hallway at Microsoft. The next day Sarah and I went to two birthday parties the second of which required costume. I went as House (from the television show House) by putting on a suit jacket and carrying a cane. Sarah wore scrubs to lend cred. to my lazy costume. Oh yeah and on Sunday Sarah bought a new car.

Investigation of a Few Application Protocols (Updated)

Windows allows for application protocols in which, through the registry, you specify a URL scheme and a command line to have that URL passed to your application. Its an easy way to hook a webbrowser up to your application. Anyone can read the doc above and then walk through the registry and pick out the application protocols but just from that info you can't tell what the application expects these URLs to look like. I did a bit of research on some of the application protocols I've seen which is listed below. Good places to look for information on URI schemes: Wikipedia URI scheme, and ESW Wiki UriSchemes.

QuickBase Formula Pretty Printer and Syntax Highlighter

Sarah asked me if I knew of a syntax highlighter for the QuickBase formula language which she uses at work. I couldn't find one but thought it might be fun to make a QuickBase Formula syntax highlighter based on the QuickBase help's description of the formula syntax. Thankfully the language is relatively simple since my skills with ANTLR, the parser generator, are rusty now and I've only used it previously for personal projects (like Javaish, the ridiculous Java based shell idea I had).

With the help of some great ANTLR examples and an ANTLR cheat sheet I was able to come up with the grammar that parses the QuickBase Formula syntax and prints out the same formula marked up with HTML SPAN tags and various CSS classes. ANTLR produces the parser in Java which I wrapped up in an applet, put in a jar, and embedded in an HTML page. The script in that page runs user input through the applet's parser and sticks the output at the bottom of the page with appropriate CSS rules to highlight and print the formula in a pretty fashion.

What I learned:

• I didn't realize that Java applets are easy to use via script in an HTML page. In the JavaScript I can simply refer to publicly exposed methods on the applet and run JavaScript strings through them. It makes for a great combination: do the heavy coding in Java and do the UI in HTML. I may end up doing this again in the future.
• I love ANTLRWorks, the ANTLR IDE, that didn't exist the last time I used ANTLR. It tells you about issues with your grammar as you create it, lets you easily debug the grammar running it forwards and backwards, display parse trees, and other useful things.

Xbox Achievements for Everyday Life

I just upgraded to the Zune 3.0 software which includes games and purchasing music on the Zune via WiFi and once again I'm thrilled that the new firmware is available for old Zunes like mine. Rooting around looking at the new features I noticed Zune Badges for the first time. They're like Xbox Achievements, for example I have a Pixies Silver Artist Power Listener award for listening to the Pixies over 1000 times. I know its ridiculous but I like it, and now I want achievements for everything.

Achievements everywhere would require more developments in self-tracking. Self-trackers, folks who keep statistics on exactly when and what they eat, when and how much they exercise, anything one may track about one's self, were the topic of a Kevin Kelly Quantified Self blog post (also check out Cory Doctorow's SF short story The Things that Make Me Weak and Strange Get Engineered Away featuring a colony of self-trackers). For someone like me with a medium length attention span the data collection needs to be completely automatic or I will lose interest and stop collecting within a week. For instance, Nike iPod shoes that keep track of how many steps the wearer takes. I'll also need software to analyze, display, and share this data on a website like Mycrocosm. I don't want to have to spend extreme amounts of time to create something as wonderful as the Feltron Report (check out his statistic on how many daily measurements he takes for the report). Once we have the data we can give out achievements for everything!

Achievements for Everyday Life

	Carnivore Eat at least ten different kinds of animals.
	Make Friends Meet at least 10% of the residents in your home town.
	Globetrotter Visit a city in every country.
	You're Old Survive at least 80 years of life.

Of course none of the above is practical yet, but how about Delicious achievements based on the public Delicious feeds? That should be doable...

Facebook Profile Views Application - Failed Idea

I had an idea for a Facebook app the other day. I wondered who actually looked at my profile and thought I could create a Facebook app that would record this information and display it. When I talked to Vishu though he said that this wasn't something that Facebook would be too happy with. Indeed the Platform Policy explicitly disallows this in section 2.8. This explained why the app didn't already exist. Its probably for the best since everyone assumes they can anonymously view Facebook profiles and would be irritated if that weren't the case.

On the topic of assumed anonymity, check out this article on the aggregation and selling off of your cell phone data including your physical location.