modi page 2 - Dave's Blog

Search
My timeline on Mastodon

The Old New Thing : Advantages of knowing your x86 machine code

2010 Feb 19, 2:27Raymond's tips for modifying x86 assembly code while debugging.PermalinkCommentstutorial debug debugging technical assembly x86 windows raymond-chen tips

WPAD Server Fiddler Extension

2010 Jan 5, 7:42

I've made a WPAD server Fiddler extension and in a fit of creativity I've named it: WPAD Server Fiddler Extension.

Of course you know about Fiddler, Eric's awesome HTTP debugger tool, the HTTP proxy that lets you inspect, visualize and modify the HTTP traffic that flows through it. And on the subject you've probably definitely heard of WPAD, the Web Proxy Auto Discovery protocol that allows web browsers like IE to use DHCP or DNS to automatically discover HTTP proxies on their network. While working on a particularly nasty WPAD bug towards the end of IE8 I really wished I had a way to see the WPAD requests and responses and modify PAC responses in Fiddler. Well the wishes of me of the past are now fulfilled by present day me as this Fiddler extension will respond to WPAD DHCP requests telling those clients (by default) that Fiddler is their proxy.

When I started working on this project I didn't really understand how DHCP worked especially with respect to WPAD. I won't bore you with my misconceptions: it works by having your one DHCP server on your network respond to regular DHCP requests as well as WPAD DHCP requests. And Windows I've found runs a DHCP client service (you can start/stop it via Start|Run|'services.msc', scroll to DHCP Client or via the command line with "net start/stop 'DHCP Client'") that caches DHCP server responses making it just slightly more difficult to test and debug my extension. If a Windows app uses the DHCP client APIs to ask for the WPAD option, this service will send out a DHCP request and take the first DHCP server response it gets. That means that if you're on a network with a DHCP server, my extension will be racing to respond to the client. If the DHCP server wins then the client ignores the WPAD response from my extension.

Various documents and tools I found useful while working on this:

PermalinkCommentsproxy fiddler http technical debug wpad pac tool dhcp

IETF: PATCH Method for HTTP

2009 Dec 18, 9:48"Several applications extending the Hypertext Transfer Protocol (HTTP) require a feature to do partial resource modification. The existing HTTP PUT method only allows a complete replacement of a document. This proposal adds a new HTTP method, PATCH, to modify an existing HTTP resource."PermalinkCommentshttp patch ietf reference via:warren technical

Just Add Johansson

2009 Dec 16, 9:41"This sort of model should remind you of 1-piece click-out toys packaged with action figures such as Teenage Mutant Ninja Turtles. Michael Johansson specializes in parodying this particular moment in toydom by creating life-sized models with that molded-plastic “break-apart and play” action specifically in mind."
PermalinkCommentsart sculpture design product commodity parody

Android eBook Reader And Makers

2009 Dec 13, 1:27

I was reading Makers, Cory Doctorow's latest novel, as it was serialized on Tor's website but with no ability to save my place within a page I set out to find a book reading app for my G1 Android phone. I stopped looking once I found Aldiko. Its got bookmarks within chapters, configurable fonts, you can look-up words in a dictionary, and has an easy method to download public domain and creative common books. I was able to take advantage of Aldiko's in-app book download system to get Makers onto my phone so I didn't have to bother with any conversion programs etc, and I didn't have to worry about spacing or layout, the book had the correct cover art, and chapter delimiters. I'm very happy with this app and finished reading Makers on it.

Makers is set in the near future and features teams of inventors, networked 3d printers, IP contention, body modifications, and Disney -- just the sort of thing you'd expect from a Cory Doctorow novel. The tale seems to be an allegory for the Internet including displacing existing businesses and the conflict between the existing big entertainment IP owners and the plethora of fans and minor content producers. The story is engaging and the characters filled out and believable. I recommend Makers and as always its Creative Commons so go take a look right now.

PermalinkCommentstor aldiko cory doctorow g1 makers ebook android book

Map/Reduce Tutorial

2009 Oct 6, 3:24The map/reduce tutorial for Hadoop the Apache open source project. "Hadoop Map/Reduce is a software framework for easily writing applications which process vast amounts of data (multi-terabyte data-sets) in-parallel on large clusters (thousands of nodes) of commodity hardware in a reliable, fault-tolerant manner."PermalinkCommentshadoop mapreduce java software programming opensource database distributed google yahoo apache technical todo

PowerShell Scanning Script

2009 Jun 27, 3:42

I've hooked up the printer/scanner to the Media Center PC since I leave that on all the time anyway so we can have a networked printer. I wanted to hook up the scanner in a somewhat similar fashion but I didn't want to install HP's software (other than the drivers of course). So I've written my own script for scanning in PowerShell that does the following:

  1. Scans using the Windows Image Acquisition APIs via COM
  2. Runs OCR on the image using Microsoft Office Document Imaging via COM (which may already be on your PC if you have Office installed)
  3. Converts the image to JPEG using .NET Image APIs
  4. Stores the OCR text into the EXIF comment field using .NET Image APIs (which means Windows Search can index the image by the text in the image)
  5. Moves the image to the public share

Here's the actual code from my scan.ps1 file:

param([Switch] $ShowProgress, [switch] $OpenCompletedResult)

$filePathTemplate = "C:\users\public\pictures\scanned\scan {0} {1}.{2}";
$time = get-date -uformat "%Y-%m-%d";

[void]([reflection.assembly]::loadfile( "C:\Windows\Microsoft.NET\Framework\v2.0.50727\System.Drawing.dll"))

$deviceManager = new-object -ComObject WIA.DeviceManager
$device = $deviceManager.DeviceInfos.Item(1).Connect();

foreach ($item in $device.Items) {
        $fileIdx = 0;
        while (test-path ($filePathTemplate -f $time,$fileIdx,"*")) {
                [void](++$fileIdx);
        }

        if ($ShowProgress) { "Scanning..." }

        $image = $item.Transfer();
        $fileName = ($filePathTemplate -f $time,$fileIdx,$image.FileExtension);
        $image.SaveFile($fileName);
        clear-variable image

        if ($ShowProgress) { "Running OCR..." }

        $modiDocument = new-object -comobject modi.document;
        $modiDocument.Create($fileName);
        $modiDocument.OCR();
        if ($modiDocument.Images.Count -gt 0) {
                $ocrText = $modiDocument.Images.Item(0).Layout.Text.ToString().Trim();
                $modiDocument.Close();
                clear-variable modiDocument

                if (!($ocrText.Equals(""))) {
                        $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $fileName
                        if (!($fileName.EndsWith(".jpg") -or $fileName.EndsWith(".jpeg"))) {
                                if ($ShowProgress) { "Converting to JPEG..." }

                                $newFileName = ($filePathTemplate -f $time,$fileIdx,"jpg");
                                $fileAsImage.Save($newFileName, [System.Drawing.Imaging.ImageFormat]::Jpeg);
                                $fileAsImage.Dispose();
                                del $fileName;

                                $fileAsImage = New-Object -TypeName system.drawing.bitmap -ArgumentList $newFileName 
                                $fileName = $newFileName
                        }

                        if ($ShowProgress) { "Saving OCR Text..." }

                        $property = $fileAsImage.PropertyItems[0];
                        $property.Id = 40092;
                        $property.Type = 1;
                        $property.Value = [system.text.encoding]::Unicode.GetBytes($ocrText);
                        $property.Len = $property.Value.Count;
                        $fileAsImage.SetPropertyItem($property);
                        $fileAsImage.Save(($fileName + ".new"));
                        $fileAsImage.Dispose();
                        del $fileName;
                        ren ($fileName + ".new") $fileName
                }
        }
        else {
                $modiDocument.Close();
                clear-variable modiDocument
        }

        if ($ShowProgress) { "Done." }

        if ($OpenCompletedResult) {
                . $fileName;
        }
        else {
                $result = dir $fileName;
                $result | add-member -membertype noteproperty -name OCRText -value $ocrText
                $result
        }
}

I ran into a few issues:

PermalinkCommentstechnical scanner ocr .net modi powershell office wia

OpenSearchDescriptionToHTML Tool

2009 Jun 10, 3:36

I've made an OpenSearchDescriptionToHTML XSLT that given an OpenSearch description file produces HTML that describes that file, lets you install it, or search with it. For example, here's a Google OpenSearch description that uses my OpenSearchDescriptionToHTML XSLT.

I had just created an OpenSearch description for WolframAlpha at work and was going about the process of adding another install link to my search provider page so that I could install it. Thinking about it, I realized I could apply an XSLT to the OpenSearch description XML to produce the HTML automatically so I wouldn't have to modify additional documents everytime I create and want to install a new OpenSearch description. While I was in there writing the XSLT I figure why not let the user try out searching with the OpenSearch description file too. And lastly I made the XSLT apply to itself to produce HTML describing its own usage.

Incidentally, I added WolframAlpha at work to replace my FileInfo search provider for the purposes of searching for information about particular Unicode characters. For instance, look at WolframAlpha's lovely output for this search for "Bopomofo zh".

PermalinkCommentstechnical xml wolframalpha opensearchdescriptiontohtml xslt opensearch

IR Ink

2009 May 29, 9:01"These inks are completely invisible to the human eye yet can be seen by using a device which can see in the infrared range - such as our modified cameras and camcorders. These inks do not fluoresce in the visible range, cannot be seen with ultraviolet lights and cannot be seen by the human eye alone." Via and not via .PermalinkCommentsir ir-paint ink purchase

Text ads in open source code will add millions in funding to OSS movement - jorgenmodin.net

2009 Apr 1, 9:26"Open source code will contain text ads in comments and variable names, and Google and other text ad companies will scan the open source code repositories and fund the projects based on what they find in the code"PermalinkCommentshumor opensource advertising code

Stop For Me, Its the Claw

2008 Sep 1, 9:35

sequelguy posted a photo:

Stop For Me, Its the Claw

Humorous modified sign: "Stop for me its the law" changed to "... its the claw" with a claw added to the stick figures arm.

PermalinkCommentsseattle washington alki watersign

How to stop an ActiveX control from running in Internet Explorer

2008 May 30, 1:36Killbits: "This article describes how to stop an ActiveX control from running in Microsoft Internet Explorer and in Windows Internet Explorer. You can do this by modifying the data value of the Compatibility Flags DWORD value for the Class identifier (CLSPermalinkCommentssecurity ie killbit msdn microsoft windows browser reference

Digging in : NTFS Time Stamps --file created in 1601, modified in 1801 and accessed in 2008!!

2008 Mar 31, 1:11Guy messes with various file timestamps on NTFS on Windows.PermalinkCommentstime timestamp ntfs windows blog article

URI Addressable Text Adventure Games

2008 Mar 2, 9:18

This post is about creating a server side z-code interpreter that represents game progress in the URI. Try it with the game Lost Pig.

I enjoy working on URIs and have the mug to prove it. Along those lines I've combined thoughts on URIs with interactive fiction. I have a limited amount of experience with Inform which generates Z-Code so I'll focus on pieces written in that. Of course we can already have URIs identifying the Z-Code files themselves, but I want URIs to identify my place in a piece of interactive fiction. The proper way to do this would be to give Z-Code its own mimetype and associate with that mimetype the format of a fragment that would contain the save state of user's interactive fiction session. A user would install a browser plugin that would generate URIs containing the appropriate fragment while you play the IF piece and be able to load URIs identifying Z-Code files and load the save state that appears in the fragment.

But all of that would be a lot of work, so I made a server side version that approximates this. On the Web Frotz Interpreter page, enter the URI of a Z-Code file to start a game. Enter your commands into the input text box at the bottom and you get a new URI after every command. For example, here's the beginning of Zork. I'm running a slightly modified version of the Unix version of Frotz. Baf's Guide to the IF Archive has lists of IF games to try out.

There are two issues with this thought, the first being the security issues with running arbitrary z-code and the second is the practical URI length limit of about 2K in IE. From the Z-Code standard and the Frotz source it looks like 'save' and 'restore' are the only commands that could do anything interesting outside of the Z-Code virtual machine. As for the length-limit on URIs I'm not sure that much can be done about that. I'm using a base64 encoded copy of the compressed input stream in the URI now. Switching to the actual save state might be smaller after enough user input.

PermalinkCommentszork frotz interactive-fiction zcode if technical uri fragment

Chumby will be cool, despite its name

2008 Feb 19, 1:51

Bedside ChumbyI signed up for the pre-release beta and purchased a Chumby last year. Chumby looks like a cousin to a GPS unit. Its similar in size with a touch screen, but has WiFi, accelerometers, and is pillow like on the sides that aren't a screen. In practice its like an Internet alarm clock that shows you photos and videos off the Web. Its hackable in that Chumby Industries tells you about the various ways to run your own stuff on the Chumby, modifying the boot sequence (it runs Linux), turning on sshd, etc, etc. The Chumby forum too has lots of info from folks who have found interesting hacks for the device.

When you turn on the Chumby it downloads and runs the latest version of the Chumby software which lets you set alarms, play music, and display Flash widgets. The Chumby website lets anyone upload their own Flash widgets to share with the community. I tried my hand at creating one using Adobe's free Flash creation SDK but I don't know Flash and didn't have the patience to learn.

Currently my Chumby is set to wake me up at 8am on weekdays with music from ShoutCast and then displays traffic and weather. At 10am everyday it switches to showing me a slide-show of LolCats. At 11pm it switches to night mode where it displays the time in dark grey text on a black background at a reduced light level so as not to disturb me while I sleep.

I like the Chumby but I have two complaints. The first is that it forces me to learn flash in order to create anything cool rather than having a built-in Web browser or depending on a more Web friendly technology. The second complaint is about its name. At first I thought the name was stupid in a kind of silly way, but now that I'm used to the name it sounds vaguely dirty.

PermalinkCommentschumby review flash linux

LA Weekly - Art+Books - Murakami REVOKed - Shelley Leopold - The Essential Online Resource for Los Angeles

2008 Feb 5, 2:02From BB's article: "In December, graffiti writers AUGER and REVOK modified a billboard advertising the wonderful Takashi Murakami exhibit ... Murakami himself saw online photos of the graffitied billboard and thought it to be "so wonderful, he had to havePermalinkCommentsart graffiti cultural-disobediance murakami via:boingboing

Old Miscellaneous Thoughts

2007 Dec 26, 5:45Miscellaneous thoughts I had that would have been relevant many months ago:
PermalinkCommentspopfly apple personal history-channel indiana-jones pipes mac technical microsoft mashup yahoo nontechnical

XSL Identity Transfom

2007 Oct 12, 4:08As noted in the XSL Transformations spec you can create the identity transform using the xsl:copy element. With the MSXML implementation of xsl:copy the example the spec gives produces slightly ugly elements. For instance given it produces . In order to ensure empty elements turn out pretty I've modified the example as follows:
    
        
            
                
                    
                
            
            
                
                    
                
            
        
    

This got me thinking about inverting XSLTs. Clearly in general an XSLT isn't invertible since an XSLT can completely ignore the input XML and produce something else entirely but then the above is an example of an XSLT that is invertible. So there is a subset of XSLTs that are invertible, how might you produce the inverse of an XSLT, and would this ever be useful?PermalinkCommentsxml msxml inverse xlst xsl

Open Library (Open Library)

2007 Jul 26, 12:12The folks at the Internet Archive have created a user modifiable Open Library that intends to catalog all books. As in all of them. Includes links to the books online (Internet Archive for ex.), where to buy (Amazon for ex.), reviews, etc.PermalinkCommentsarchive library opensource wiki research book books literature catalog reference

Backup Notes

2007 Jul 13, 8:30I bought an external backup drive a few weekends ago. I've previously setup a Subversion repository so I decided to move everything into the repository and then back it up. So in went the contents of all of my %USERPROFILE% and ~ directories with a bit of sorting and pruning. Not too much though given its much easier to dump in everything and search for what I want then to take the time to examine and grade each file. What follows are the notes I took while setting this up. It takes me a bit of time to look up the help on each command so I figure I'll write it all down here for the benefit of myself and potentially others...

Setting Up the Backup Drive For Linux
I first changed the filesystem on the drive to ext3. I plugged it into my USB2.0 port and ran fdisk:

sudo fdisk /dev/sda

Useful commands I used to do this follow mostly in order:
m
help
p
print current partitions
d
delete current partition
n
create new partition (I used the defaults)
w
write changes and exit
Then I formatted for ext3.

sudo mkfs.ext3 /dev/sda1

I made it easy to mount:

sudo vim /etc/fstab
# added line to end:
/dev/sda1 /media/backup ext3 rw,user,noauto 0 0

I setup the directory structure on the disk

mount /media/backup
sudo mkdir /media/backup/users
sudo mkdir /media/backup/users/dave
sudo chown dave:dave /media/backup/users/dave


After all that its easy to make a copy of the Subversion repository:

mount /media/backup
cp -Rv /home/dave/svn /media/backup/users/dave/
umount /media/backup

Next on the agenda is to add a cron job to do this regularly.

Subversion Command Reference
On a machine that has local access to the Subversion repository you can check out a specific subdirectory as follows using the file scheme:

svn co file:///home/dave/svn/trunk/web/dave%40deletethis.net/public_html

Note also that although one of my directories is named 'dave@deletethis.net' Subversion requires the '@' to be percent-encoded.
Other useful subversion commands:
svn help
help
svn list file:///home/dave/svn/
list all files in root dir of svn depot
svn list -R file:///home/dave/svn/
list all files in svn depot
svn list -R file:///home/dave/svn/ | grep \/$
list all directories
svn status
List status of all files in the working copy directory as in - modified, not in repository, etc
svn update
Brings the working copy up to date wrt the repository
svn commit
Commit changes from the working copy to the repository
svn add / move / delete
Perform the specified action -- occurs immediately


Setting up Windows Client for Auto Auth into SVN
When using an SVN client on Windows via svn+ssh its useful to have the Windows automatically generate connections to the SVN server. I use putty on my Windows machines so I read the directions on using public keys with putty.

putty.exe dave@deletethis.net
cd .ssh
vim authorized_keys # leave the putty window open for now
puttygen.exe
Click the 'generate' button
Move the mouse around until finished
Copy text in 'Public key for pasting into OpenSSH authorized_keys file:' to putty window & save & close putty window
Enter Key passphrase & Comment in puttygen
Save the private key somewhere private
pageant.exe
'Add Key' the private key just saved.



Checking out using Tortoise SVN
On one of my Windows machines I've already installed Tortoise SVN. Checking out from my SVN repository was really easy. I just right clicked in Explorer in a directory and selected "SVN Checkout...". Then in the following dialog I entered the svn URI:

svn+ssh://dave@deletethis.net/home/dave/svn/trunk/web/dave%40deletethis.net/public_html/

Note again that the '@' that is part of the directory name is percent-encoded as '%40' while the '@' in the userinfo is not.

Windows Command Line Check Out
On my media center I didn't want to install Tortoise SVN so rather I used the command line tool. I setup pageant like before the only difficulty was getting the SVN command line tool to use putty. With the default configuration you can use the SVN_SSH environment variable to point at a compliant SSH command line tool. The trick is that its interpreted as a backslash escaped string. So I set mine thusly:

set SVN_SSH=C:\\users\\dave\\bin\\putty\\plink.exe

The escaping solved the vague error I received about not being able to create the tunnel.PermalinkCommentsbackup technical personal windows svn linux subversion
Older EntriesNewer Entries Creative Commons License Some rights reserved.