network page 3 - Dave's Blog

WPAD Server Fiddler Extension

I've made a WPAD server Fiddler extension and in a fit of creativity I've named it: WPAD Server Fiddler Extension.

Of course you know about Fiddler, Eric's awesome HTTP debugger tool, the HTTP proxy that lets you inspect, visualize and modify the HTTP traffic that flows through it. And on the subject you've probably definitely heard of WPAD, the Web Proxy Auto Discovery protocol that allows web browsers like IE to use DHCP or DNS to automatically discover HTTP proxies on their network. While working on a particularly nasty WPAD bug towards the end of IE8 I really wished I had a way to see the WPAD requests and responses and modify PAC responses in Fiddler. Well the wishes of me of the past are now fulfilled by present day me as this Fiddler extension will respond to WPAD DHCP requests telling those clients (by default) that Fiddler is their proxy.

When I started working on this project I didn't really understand how DHCP worked especially with respect to WPAD. I won't bore you with my misconceptions: it works by having your one DHCP server on your network respond to regular DHCP requests as well as WPAD DHCP requests. And Windows I've found runs a DHCP client service (you can start/stop it via Start|Run|'services.msc', scroll to DHCP Client or via the command line with "net start/stop 'DHCP Client'") that caches DHCP server responses making it just slightly more difficult to test and debug my extension. If a Windows app uses the DHCP client APIs to ask for the WPAD option, this service will send out a DHCP request and take the first DHCP server response it gets. That means that if you're on a network with a DHCP server, my extension will be racing to respond to the client. If the DHCP server wins then the client ignores the WPAD response from my extension.

Various documents and tools I found useful while working on this:

• WPAD IETF draft document the never out of draft document describing WPAD
• DHCP RFC
• DHCP options RFC
• DHCP Wikipedia article which includes helpful byte layout tables for DHCP messages.
• Fiddler Wiki esp. the dev section but also the portion on the new preferences API.
• Fiddler Dev Docs and for those things not covered in the Fiddler docs, Visual Studio's name completion (or the .NET Reflector) usually served me well.
• Netmon a great network sniffing tool for all my non-HTTP network inspection needs.
• Tiny DHCP Server on CodeProject which implements a conventional DHCP server.

Android eBook Reader And Makers

I was reading Makers, Cory Doctorow's latest novel, as it was serialized on Tor's website but with no ability to save my place within a page I set out to find a book reading app for my G1 Android phone. I stopped looking once I found Aldiko. Its got bookmarks within chapters, configurable fonts, you can look-up words in a dictionary, and has an easy method to download public domain and creative common books. I was able to take advantage of Aldiko's in-app book download system to get Makers onto my phone so I didn't have to bother with any conversion programs etc, and I didn't have to worry about spacing or layout, the book had the correct cover art, and chapter delimiters. I'm very happy with this app and finished reading Makers on it.

Makers is set in the near future and features teams of inventors, networked 3d printers, IP contention, body modifications, and Disney -- just the sort of thing you'd expect from a Cory Doctorow novel. The tale seems to be an allegory for the Internet including displacing existing businesses and the conflict between the existing big entertainment IP owners and the plethora of fans and minor content producers. The story is engaging and the characters filled out and believable. I recommend Makers and as always its Creative Commons so go take a look right now.

Comcast Digital Switch Impact on My Windows Media Center

Irritatingly out of line with what their commercials say, in my area Comcast, under the covers of the national broadcast digital switch, is sneaking in their own switch to digital, moving channels above 30 to their own digital format. Previously, I had Windows 7 Media Center running on a PC with a Hauppauge PVR500 which can decode two television signals at once setup to record shows I like. The XBox 360 works great as a Media Center client letting me easily watch the recorded shows over my home network on my normal TV.

Unfortunately with Comcast's change, now one needs a cable box or a Comcast digital to analog converter in order to view their signal, but Comcast is offering up to two free converters for those who'd like them. The second of my two free converters I hooked up to the Media Center PC and I got the IR Blaster that came with my Hauppauge out of the garage. I plugged in the USB IR Blaster to my PC, connected one of the IR transmitters to the 1st port on the IR Blaster, and sat the IR transmitter next to the converter's IR receiver. I went through the Media Center TV setup again and happily it was able to figure out how to correctly change the channel on the converter. So I can record now, however:

1. I can only record one thing at a time now
2. Changing the channel is slow taking many seconds (no flipping through channels for me)
3. The Hauppauge card can't know if the channel change worked. So if it tries to change to HBO (I get it for free with one of the Comcast packages) which is encrypted and the converted won't show, the channel doesn't change but the PC doesn't know it and ends up recording some other channel.

PowerShell Scanning Script

I've hooked up the printer/scanner to the Media Center PC since I leave that on all the time anyway so we can have a networked printer. I wanted to hook up the scanner in a somewhat similar fashion but I didn't want to install HP's software (other than the drivers of course). So I've written my own script for scanning in PowerShell that does the following:

1. Scans using the Windows Image Acquisition APIs via COM
2. Runs OCR on the image using Microsoft Office Document Imaging via COM (which may already be on your PC if you have Office installed)
3. Converts the image to JPEG using .NET Image APIs
4. Stores the OCR text into the EXIF comment field using .NET Image APIs (which means Windows Search can index the image by the text in the image)
5. Moves the image to the public share

Here's the actual code from my scan.ps1 file:

param([Switch] $ShowProgress, [switch] $OpenCompletedResult)

$filePathTemplate = "C:\users\public\pictures\scanned\scan {0} {1}.{2}";
$time = get-date -uformat "%Y-%m-%d";

[void]([reflection.assembly]::loadfile( "C:\Windows\Microsoft.NET\Framework\v2.0.50727\System.Drawing.dll"))

$deviceManager = new-object -ComObject WIA.DeviceManager
$device = $deviceManager.DeviceInfos.Item(1).Connect();

foreach ($item in $device.Items) {
        $fileIdx = 0;
        while (test-path ($filePathTemplate -f $time,$fileIdx,"*")) {
                [void](++$fileIdx);
        }

        if ($ShowProgress) { "Scanning..." }

        $image = $item.Transfer();
        $fileName = ($filePathTemplate -f $time,$fileIdx,$image.FileExtension);
        $image.SaveFile($fileName);
        clear-variable image

        if ($ShowProgress) { "Running OCR..." }

        $modiDocument = new-object -comobject modi.document;
        $modiDocument.Create($fileName);
        $modiDocument.OCR();
        if ($modiDocument.Images.Count -gt 0) {
                $ocrText = $modiDocument.Images.Item(0).Layout.Text.ToString().Trim();
                $modiDocument.Close();
                clear-variable modiDocument

                if (!($ocrText.Equals(""))) {
                        $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $fileName
                        if (!($fileName.EndsWith(".jpg") -or $fileName.EndsWith(".jpeg"))) {
                                if ($ShowProgress) { "Converting to JPEG..." }

                                $newFileName = ($filePathTemplate -f $time,$fileIdx,"jpg");
                                $fileAsImage.Save($newFileName, [System.Drawing.Imaging.ImageFormat]::Jpeg);
                                $fileAsImage.Dispose();
                                del $fileName;

                                $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $newFileName 
                                $fileName = $newFileName
                        }

                        if ($ShowProgress) { "Saving OCR Text..." }

                        $property = $fileAsImage.PropertyItems[0];
                        $property.Id = 40092;
                        $property.Type = 1;
                        $property.Value = [system.text.encoding]::Unicode.GetBytes($ocrText);
                        $property.Len = $property.Value.Count;
                        $fileAsImage.SetPropertyItem($property);
                        $fileAsImage.Save(($fileName + ".new"));
                        $fileAsImage.Dispose();
                        del $fileName;
                        ren ($fileName + ".new") $fileName
                }
        }
        else {
                $modiDocument.Close();
                clear-variable modiDocument
        }

        if ($ShowProgress) { "Done." }

        if ($OpenCompletedResult) {
                . $fileName;
        }
        else {
                $result = dir $fileName;
                $result | add-member -membertype noteproperty -name OCRText -value $ocrText
                $result
        }
}

I ran into a few issues:

• MODI doesn't seem to be in the Office 2010 Technical Preview I installed first. Installing Office 2007 fixed that.
• The MODI.Document class, at least via PowerShell, can't be instantiated in a 64bit environment. To run the script on my 64bit OS I had to start powershell from the 32bit cmd.exe (C:\windows\syswow64\cmd.exe).
• I was planning to hook up my script to the scanner's 'Scan' button, but HP didn't get the button working for their Vista driver. Their workaround is "don't do that!".
• You must call Image.Dispose() to get .NET to release its reference to the corresponding image file.
• In trying to figure out how to store the text in the files comment, I ran into a dead-end trying to find the corresponding setter for GetDetailsOf which folks like James O'Neil use in PowerShell for interesting ends.

A Brief History of Microsoft's Live Search's New Domain Bing

• 2003 July: The first archived version of bing.com features "bing! is a small device (e.g., possibly even a small Band-Aid(R)-like sticker!) that vibrates when a person's cell phone rings." I can't recall 2003 cell phones, were they big enough to require this device?
• 2004 August: Site for the same device is rewritten and looks much better, IMHO.
• 2006 June: The domain is now parked by easyDNS. I guess the "bing!" device didn't work out?
• 2006 November: Its now "BING*" and they won't say what they're working on ("we're still in stealth mode") but they are hiring C#/.NET developers.
• 2007 January: And they're gone. Without even exiting stealth mode. Too bad, I liked their logo. Their domain is now for sale...
• 2007 February: Looks like EasyMail buys the domain and offers a physical mailing service in Australia: "By simply clicking a button on your computer, mail is beamed electronically to a bing post office. Your mail is automatically printed, folded, enveloped and dispatched into the Australia Post network the very same day."
• Present: Now its the new home for Live Search of course.

The new name reminds me of the show Friends. Also, I hope they get a new favicon - I don't enjoy the stretched 'b' nor its color scheme.