vs page 4 - Dave's Blog

IE8 Search Providers, Accelerators, and Local Applications Hack

There's no easy way to use local applications on a PC as the result of an accelerator or a search provider in IE8 but there is a hack-y/obvious way, that I'll describe here. Both accelerators and search providers in IE8 fill in URL templates and navigate to the resulting URL when an accelerator or search provider is executed by the user. These URLs are limited in scheme to http and https but those pages may do anything any other webpage may do. If your local application has an ActiveX control you could use that, or (as I will provide examples for) if the local application has registered for an application protocol you can redirect to that URL. In any case, unfortunately this means that you must put a webpage on the Internet in order to get an accelerator or search provider to use a local application.

For examples of the app protocol case, I've created a callto accelerator that uses whatever application is registered for the callto scheme on your system, and a Windows Search search provider that opens Explorer's search with your search query. The callto accelerator navigates to my redirection page with 'callto:' followed by the selected text in the fragment and the redirection page redirects to that callto URL. In the Windows Search search provider case the same thing happens except the fragment contains 'search-ms:query=' followed by the selected text, which starts Windows Search on your system with the selected text as the query. I've looked into app protocols previously.

Baseball Game Friday, House Things Previously

On Friday Sarah and I went to the Mariners vs Rangers game at Safeco Field with Eric and Jane. The Mariners lost but then before the game the announcement made outside the stadium guaranteed the best service and a good time, not a winning game -- and they were right about the good time.

The night before, we saw The Hangover which was very funny and included Zach Galifianakis who was great. Incidentally, take a look at some of Zach's Between Two Ferns.

Last weekend Sarah and I mounted the TV to the wall which was exciting and we saw a mouse in the house!

Mariners vs Rangers, Safeco Field, Seattle

sequelguy posted a photo:

PowerShell Scanning Script

I've hooked up the printer/scanner to the Media Center PC since I leave that on all the time anyway so we can have a networked printer. I wanted to hook up the scanner in a somewhat similar fashion but I didn't want to install HP's software (other than the drivers of course). So I've written my own script for scanning in PowerShell that does the following:

1. Scans using the Windows Image Acquisition APIs via COM
2. Runs OCR on the image using Microsoft Office Document Imaging via COM (which may already be on your PC if you have Office installed)
3. Converts the image to JPEG using .NET Image APIs
4. Stores the OCR text into the EXIF comment field using .NET Image APIs (which means Windows Search can index the image by the text in the image)
5. Moves the image to the public share

Here's the actual code from my scan.ps1 file:

param([Switch] $ShowProgress, [switch] $OpenCompletedResult)

$filePathTemplate = "C:\users\public\pictures\scanned\scan {0} {1}.{2}";
$time = get-date -uformat "%Y-%m-%d";

[void]([reflection.assembly]::loadfile( "C:\Windows\Microsoft.NET\Framework\v2.0.50727\System.Drawing.dll"))

$deviceManager = new-object -ComObject WIA.DeviceManager
$device = $deviceManager.DeviceInfos.Item(1).Connect();

foreach ($item in $device.Items) {
        $fileIdx = 0;
        while (test-path ($filePathTemplate -f $time,$fileIdx,"*")) {
                [void](++$fileIdx);
        }

        if ($ShowProgress) { "Scanning..." }

        $image = $item.Transfer();
        $fileName = ($filePathTemplate -f $time,$fileIdx,$image.FileExtension);
        $image.SaveFile($fileName);
        clear-variable image

        if ($ShowProgress) { "Running OCR..." }

        $modiDocument = new-object -comobject modi.document;
        $modiDocument.Create($fileName);
        $modiDocument.OCR();
        if ($modiDocument.Images.Count -gt 0) {
                $ocrText = $modiDocument.Images.Item(0).Layout.Text.ToString().Trim();
                $modiDocument.Close();
                clear-variable modiDocument

                if (!($ocrText.Equals(""))) {
                        $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $fileName
                        if (!($fileName.EndsWith(".jpg") -or $fileName.EndsWith(".jpeg"))) {
                                if ($ShowProgress) { "Converting to JPEG..." }

                                $newFileName = ($filePathTemplate -f $time,$fileIdx,"jpg");
                                $fileAsImage.Save($newFileName, [System.Drawing.Imaging.ImageFormat]::Jpeg);
                                $fileAsImage.Dispose();
                                del $fileName;

                                $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $newFileName 
                                $fileName = $newFileName
                        }

                        if ($ShowProgress) { "Saving OCR Text..." }

                        $property = $fileAsImage.PropertyItems[0];
                        $property.Id = 40092;
                        $property.Type = 1;
                        $property.Value = [system.text.encoding]::Unicode.GetBytes($ocrText);
                        $property.Len = $property.Value.Count;
                        $fileAsImage.SetPropertyItem($property);
                        $fileAsImage.Save(($fileName + ".new"));
                        $fileAsImage.Dispose();
                        del $fileName;
                        ren ($fileName + ".new") $fileName
                }
        }
        else {
                $modiDocument.Close();
                clear-variable modiDocument
        }

        if ($ShowProgress) { "Done." }

        if ($OpenCompletedResult) {
                . $fileName;
        }
        else {
                $result = dir $fileName;
                $result | add-member -membertype noteproperty -name OCRText -value $ocrText
                $result
        }
}

I ran into a few issues:

• MODI doesn't seem to be in the Office 2010 Technical Preview I installed first. Installing Office 2007 fixed that.
• The MODI.Document class, at least via PowerShell, can't be instantiated in a 64bit environment. To run the script on my 64bit OS I had to start powershell from the 32bit cmd.exe (C:\windows\syswow64\cmd.exe).
• I was planning to hook up my script to the scanner's 'Scan' button, but HP didn't get the button working for their Vista driver. Their workaround is "don't do that!".
• You must call Image.Dispose() to get .NET to release its reference to the corresponding image file.
• In trying to figure out how to store the text in the files comment, I ran into a dead-end trying to find the corresponding setter for GetDetailsOf which folks like James O'Neil use in PowerShell for interesting ends.

Flickr Visual Search in IE8

A while ago I promised to say how an xsltproc Meddler script would be useful and the general answer is its useful for hooking up a client application that wants data from the web in a particular XML format and the data is available on the web but in another XML format. The specific case for this post is a Flickr Search service that includes IE8 Visual Search Suggestions. IE8 wants the Visual Search Suggestions XML format and Flickr gives out search data in their Flickr web API XML format.

So I wrote an XSLT to convert from Flickr Search XML to Visual Suggestions XML and used my xsltproc Meddler script to actually apply this xslt.

After getting this all working I've placed the result in two places: (1) I've updated the xsltproc Meddler script to include this XSLT and an XML file to install it as a search provider - although you'll need to edit the XML to include your own Flickr API key. (2) I've created a service for this so you can just install the Flickr search provider if you're interested in having the functionality and don't care about the implementation. Additionally, to the search provider I've added accelerator preview support to show the Flickr slideshow which I think looks snazzy.

Doing a quick search for this it looks like there's at least one other such implementation, but mine has the distinction of being done through XSLT which I provide, updated XML namespaces to work with the released version of IE8, and I made it so you know its good.

Awesome Spokesmen Billy Mays and Vince Offer

There can be no doubt that infomercial pitchmen Billy Mays and Vince Offer are awesome, but how can I decide which is my favorite?

Infomercial Performance.

Among other things they sell Zorbeez and ShamWow respectively, both of which are absorbant towels so first up I can get a somewhat apples-to-apples comparison on informercials for comparable products. Thankfully someone has created a Vince Offer vs Billy Mays youtube video featuring these informercials.

Billy has an upbeat yell to everything he says while Vince has an endearing condescension to his "You followin' me camera guy?" In the end I must give this one to Vince Offer.

Extracurriculars.

Billy Mays has a new show Pitchmen on the Discovery Channel, a reality show involving folks pitching inventions to the pitchmen Billy Mays and someone else. Vince Offer was arrested for assaulting a prostitute. A clear win for Billy.

Intra-pitchman Interactions.

Billy has challenged Vince to a pitchman-off which on the surface sounds like a point for Billy, but if you listen to the challenge Billy says that Vince attempted to extend the olive branch. Billy Mays calls out Vince on the Adam Corolla Show

Outline View Internet Explorer Extension

I've made another extension for IE8, Outline View, which gives you a side bar in IE that displays an outline of the current page and lets you make intrapage bookmarks.

The outline is generated based on the heading tags in the document (e.g. h1, h2, etc), kind of like what W3C's Semantic data extractor tool displays for an outline. So if the page doesn't use heading tags the way the HTML spec intended or just sticks img tags in them, then the outline doesn't look so hot. On a page that does use headings as intended though it looks really good. For instance a section from the HTML 4 spec shows up quite nicely and I find its actually useful to be able to jump around to the different sections. Actually, I've been surprised going to various blogs how well the outline view is actually working -- I thought a lot more webdevs would be abusing their heading tags.

I've also added intrapage bookmarks. When you make a text selection and clear it, that selected text is added as a temporary intrapage bookmark which shows up in the correct place in the outline. You can navigate to the bookmark or right click to make it permanent. Right now I'm storing the permanent intrapage bookmarks in IE8's new per-domain DOM storage because I wanted to avoid writing code to synchronize a cross process store of bookmarks, it allowed me to play with the DOM storage a bit, and the bookmarks will get cleared appropriately when the user clears their history via the control panel.

Notes on Creating Internet Explorer Extensions in C++ and COM

Working on Internet Explorer extensions in C++ & COM, I had to relearn or rediscover how to do several totally basic and important things. To save myself and possibly others trouble in the future, here's some pertinent links and tips.

First you must choose your IE extensibility point. Here's a very short list of the few I've used:

• Explorer Bars and IE Toolbars
• IE Tools, IE Help, and IE Context Menu menu items
• IE Toolbar Buttons

Once you've created your COM object that implements IObjectWithSite and whatever other interfaces your extensibility point requires as described in the above links you'll see your SetSite method get called by IE. You might want to know how to get the top level browser object from the IUnknown site object passed in via that method.

After that you may also want to listen for some events from the browser. To do this you'll need to:

1. Implement the dispinterface that has the event you want. For instance DWebBrowserEvents2, or HTMLDocumentEvents, or HTMLWindowEvents2. You'll have to search around in that area of the documentation to find the event you're looking for.
2. Register for events using AtlAdvise. The object you need to subscribe to depends on the events you want. For example, DWebBrowserEvents2 come from the webbrowser object, HTMLDocumentEvents come from the document object assuming its an HTML document (I obtained via get_Document method on the webbrowser), and HTMLWindowEvents2 come from the window object (which oddly I obtained via calling the get_script method on the document object). Note that depending on when your SetSite method is called the document may not exist yet. For my extension I signed up for browser events immediately and then listened for events like NavigateComplete before signing up for document and window events.
3. Implement IDispatch. The Invoke method will get called with event notifications from the dispinterfaces you sign up for in AtlAdvise. Implementing Invoke manually is a slight pain as all the parameters come in as VARIANTs and are in reverse order. There's some ATL macros that may make this easier but I didn't bother.
4. Call AtlUnadvise at some point -- at the latest when SetSite is called again and your site object changes.

If you want to check if an IHTMLElement is not visible on screen due how the page is scrolled, try comparing the Body or Document Element's client height and width, which appears to be the dimensions of the visible document area, to the element's bounding client rect which appears to be its position relative to the upper left corner of the visible document area. I've found this to be working for me so far, but I'm not positive that frames, iframes, zooming, editable document areas, etc won't mess this up.

Be sure to use pointers you get from the IWebBrowser/IHTMLDocument/etc. only on the thread on which you obtained the pointer or correctly marshal the pointers to other threads to avoid weird crashes and hangs.

Obtaining the HTML document of a subframe is slightly more complicated then you might hope. On the other hand this might be resolved by the new to IE8 method IHTMLFrameElement3::get_contentDocument

Check out Eric's IE blog post on IE extensibility which has some great links on this topic as well.

Investigation of a Few Application Protocols (Updated)

Windows allows for application protocols in which, through the registry, you specify a URL scheme and a command line to have that URL passed to your application. Its an easy way to hook a webbrowser up to your application. Anyone can read the doc above and then walk through the registry and pick out the application protocols but just from that info you can't tell what the application expects these URLs to look like. I did a bit of research on some of the application protocols I've seen which is listed below. Good places to look for information on URI schemes: Wikipedia URI scheme, and ESW Wiki UriSchemes.