text page 8 - Dave's Blog

Which which - Batch File Hackiness

2007 Aug 9, 5:41To satisfy my hands which have already learned to type *nix commands I like to install Win32 versions of common GNU utilities. Unfortunately, the which command is a rather literal port and requires you to enter the entire name of the command for which you're looking. That is 'which which' won't find itself but 'which which.exe' will. This makes this almost useless for me so I thought to write my own as a batch file. I had learned about a few goodies available in cmd.exe that I thought would make this an easy task. It turned out to be more difficult than I thought.

for /F "usebackq tokens=*" %%a in ( `"echo %PATH:;=& echo %"` ) do (
    for /F "usebackq tokens=*" %%b in ( `"echo %PATHEXT:;=& echo %"` ) do (
        if exist "%%a"\%1%%b (
            for  %%c in ( "%%a"\%1%%b ) do (
                echo %%~fc
            )
        )
    )
)

The environment variables PATH and PATHEXT hold the list of paths to search through to find commands, and the extensions of files that should be run as commands respectively. The 'for /F "usebackq tokens=*" %%a in (...) do (...)' runs the 'do' portion with %%a sequentially taking on the value of every line in the 'in' portion. That's nice, but PATH and PATHEXT don't have their elements on different lines and I don't know of a way to escape a newline character to appear in a batch file. In order to get the PATH and PATHEXT's elements onto different lines I used the %ENV:a=b% syntax which replaces occurrences of a with b in the value of ENV. I replaced the ';' delimiter with the text '& echo ' which means %PATHEXT:;=& echo% evaluates to something like "echo .COM& echo .EXE& echo .BAT& ...". I have to put the whole expression in double quotes in order to escape the '&' for appearing in the batch file. The usebackq and the backwards quotes means that the backquoted string should be replaced with the output of the execution of its content. So in that fashion I'm able to get each element of the env. variable onto new lines. The rest is pretty straight forward.

Also, it supports wildcards:

C:\Users\davris>which.cmd *hi*
C:\Windows\System32\GRAPHICS.COM
C:\Windows\System32\SearchIndexer.exe
D:\bin\which.exe
D:\bin\which.cmd

which cmd technical batch for

Two-for Script File

2007 Aug 6, 5:40I was messing with the XSLT to XSL Converter source which is a javascript file that can be run with cscript.exe. I've changed it to be like a very basic version of xsltproc that simply runs an XML file through an XSLT. I also wanted to run this from the command prompt without writing "cscript ..." everytime. I decided to make like perl programmers I've seen and make a JS file that works as a batch file and a JS file at the same time.

Here's a basic version of what I ended doing applied to a 'hello world' script named helloworld.cmd:

/* 2> NUL
@echo off
cscript /e:javascript /nologo "%~f0" %*
@goto :eof

    Hello World
        Says 'Hello world.' when you run it.
*/

var outText = 'Hello world.';
WScript.Echo(outText);

Running this on a command prompt gives the following:

C:\Users\davris>helloworld

C:\Users\davris>/*  2>NUL
Hello world.

However, after a little more experimentation I found this was slightly overkill for my purposes since if I rename the file to helloworld.js and just type its name like a command it is run by cscript:

C:\Users\davris>helloworld
Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

Hello world.

So this time I didn't need all that but if ever in the future I need to run a batch file then a JS file I can do it with one file...cmd js technical cscript batch xslt xsl javascript

Backup Notes

2007 Jul 13, 8:30I bought an external backup drive a few weekends ago. I've previously setup a Subversion repository so I decided to move everything into the repository and then back it up. So in went the contents of all of my %USERPROFILE% and ~ directories with a bit of sorting and pruning. Not too much though given its much easier to dump in everything and search for what I want then to take the time to examine and grade each file. What follows are the notes I took while setting this up. It takes me a bit of time to look up the help on each command so I figure I'll write it all down here for the benefit of myself and potentially others...

Setting Up the Backup Drive For Linux
I first changed the filesystem on the drive to ext3. I plugged it into my USB2.0 port and ran fdisk:

sudo fdisk /dev/sda

Useful commands I used to do this follow mostly in order:

m

help

p

print current partitions

d

delete current partition

n

create new partition (I used the defaults)

w

write changes and exit

Then I formatted for ext3.

sudo mkfs.ext3 /dev/sda1

I made it easy to mount:

sudo vim /etc/fstab
# added line to end:
/dev/sda1 /media/backup ext3 rw,user,noauto 0 0

I setup the directory structure on the disk

mount /media/backup
sudo mkdir /media/backup/users
sudo mkdir /media/backup/users/dave
sudo chown dave:dave /media/backup/users/dave

After all that its easy to make a copy of the Subversion repository:

mount /media/backup
cp -Rv /home/dave/svn /media/backup/users/dave/
umount /media/backup

Next on the agenda is to add a cron job to do this regularly.

Subversion Command Reference
On a machine that has local access to the Subversion repository you can check out a specific subdirectory as follows using the file scheme:

svn co file:///home/dave/svn/trunk/web/dave%40deletethis.net/public_html

Note also that although one of my directories is named 'dave@deletethis.net' Subversion requires the '@' to be percent-encoded.
Other useful subversion commands:

svn help

help

svn list file:///home/dave/svn/

list all files in root dir of svn depot

svn list -R file:///home/dave/svn/

list all files in svn depot

svn list -R file:///home/dave/svn/ | grep \/$

list all directories

svn status

List status of all files in the working copy directory as in - modified, not in repository, etc

svn update

Brings the working copy up to date wrt the repository

svn commit

Commit changes from the working copy to the repository

svn add / move / delete

Perform the specified action -- occurs immediately

Setting up Windows Client for Auto Auth into SVN
When using an SVN client on Windows via svn+ssh its useful to have the Windows automatically generate connections to the SVN server. I use putty on my Windows machines so I read the directions on using public keys with putty.

putty.exe dave@deletethis.net
cd .ssh
vim authorized_keys # leave the putty window open for now
puttygen.exe
Click the 'generate' button
Move the mouse around until finished
Copy text in 'Public key for pasting into OpenSSH authorized_keys file:' to putty window & save & close putty window
Enter Key passphrase & Comment in puttygen
Save the private key somewhere private
pageant.exe
'Add Key' the private key just saved.

Checking out using Tortoise SVN
On one of my Windows machines I've already installed Tortoise SVN. Checking out from my SVN repository was really easy. I just right clicked in Explorer in a directory and selected "SVN Checkout...". Then in the following dialog I entered the svn URI:

svn+ssh://dave@deletethis.net/home/dave/svn/trunk/web/dave%40deletethis.net/public_html/

Note again that the '@' that is part of the directory name is percent-encoded as '%40' while the '@' in the userinfo is not.

Windows Command Line Check Out
On my media center I didn't want to install Tortoise SVN so rather I used the command line tool. I setup pageant like before the only difficulty was getting the SVN command line tool to use putty. With the default configuration you can use the SVN_SSH environment variable to point at a compliant SSH command line tool. The trick is that its interpreted as a backslash escaped string. So I set mine thusly:

set SVN_SSH=C:\\users\\dave\\bin\\putty\\plink.exe

The escaping solved the vague error I received about not being able to create the tunnel.backup technical personal windows svn linux subversion

Second Life Translator

2007 Jul 4, 10:58Hackdiary
I really enjoy reading Matt Biddulph's blog hackdiary. An entry some time ago talked about his Second Life flickr screen which is a screen in Second Life that displays images from flickr.com based on viewers suggested tags. I'm a novice to the Second Life scripting API and so it was from this blog post I became aware of the llHTTPRequest. This is like the XMLHttpRequest for Second Life code in that it lets you make HTTP requests. I decided that I too could do something cool with this.

Translator
I decided to make a translator object that a Second Life user would wear that would translate anything said near them. The details aren't too surprising: The translator object keeps an owner modifiable list of translation instructions each consisting of who to listen to, the language they speak, who to tell the translation to, and into what language to translate. When the translator hears someone, it runs through its list of translation instructions and when it finds a match for the speaker uses the llHTTPRequest to send off what was said to Google translate. When the result comes back the translator simply says the response.

Issues
Unfortunately, the llHTTPRequest limits the response size to 2K and no translation site I can find has the translated text in the first 2K. There's a flag HTTP_BODY_MAXLENGTH provided but it defaults to 2K and you can't change its value. So I decided to setup a PHP script on my site to act as a translating proxy and parse the translated text out of the HTML response from Google translate. Through experimentation I found that their site can take parameters text and langpair queries in the query like so: http://translate.google.com/translate_t?text=car%20moi%20m%C3%AAme%20j%27en%20rit&langpair=fr|en. On the topic of non US-ASCII characters (which is important for a translator) I found that llHTTPRequest encodes non US-ASCII characters as percent-encoded UTF-8 when constructing the request URI. However, when Google translate takes parameters off the URI it only seems to interpret it as percent-encoded UTF-8 when the user-agent is IE's. So after changing my PHP script to use IE7's user-agent non US-ASCII character input worked.

In Use
Actually using it in practice is rather difficult. Between typos, slang, abbreviations, and the current state of the free online translators its very difficult to carry on a conversation. Additionally, I don't really like talking to random people on Second Life anyway. So... not too useful.personal translate second-life technical translator sl code google php llhttprequest

Resource Tools - IE7 XML Source View Upgrade Part 1

2007 May 9, 4:15I read about text/xml URI fragment resolution a few months ago. I was interested to find another kind of fragment reference other than the text/html URI fragment but of course I didn't find an implementation in IE, Firefox, or Opera. I decided to see how much work would be required to implement this in IE.

In IE and Firefox when you open an XML file that doesn't have an XML stylesheet the XML source is rendered with syntax highlighting. In IE I also noticed that the gold bar appears when you open an XML file off of your local machine. To me this suggested that the XML source was being rendered as HTML which I assumed was produced by running an XSLT on the source XML file. If so, I figured I could modify the XSLT to implement text/xml URI fragments. I ran FileMon to see if iexplore.exe loaded an XSLT file when opening an XML file. Only the XML file and MSXML3.DLL were opened and no XSLTs were loaded as files. My next hope for modifying the XSLT was if it existed as a resource in MSXML3.DLL. I did a findstr on the DLL for SCRIPT and found an XSLT so I decided to check for resources in MSXML3.DLL. Unfortunately my previous resource viewer didn't work correctly so I decided to write my own.

I created resource tools to view and modify resources in Windows modules. The viewer outputs HTML with links to the individual resources of a module using the res URI scheme that's built into IE. The modifier is a simple command line tool that replaces or adds one resource at a time to a module.

Using these tools I found that the XSLT was stored as a resource in MSXML3.DLL. I'll talk more about the existing XSLT and the one I replaced it with next time.resource technical xml msxml res xslt xsl

Delicious shortcut tag

2007 Apr 8, 3:05Shortcut Tag?
I just saw this on another user's delicious links: a link to ESV search that's tagged with, among other things, "shortcut:esv". When viewed on del.icio.us there's a text box that lets you search using that link. I hadn't seen this before, but it seems pretty cool and I'm surprised I hadn't seen it previously. A delicious post with such a tag ends up looking like the following: I tried searching for information on this and I've found other delicious users doing the same thing, but nothing about the tag itself. If you know any information especially official information from del.icio.us itself please post links in reply to this post. So without further preface here's what I've learned about the del.icio.us shortcut tag.

How-to
To get a search box in your del.icio.us links make a post that satisfies the following requirements:

1. One of the tags must begin with the text 'shortcut:'. You can have more text following that in the tag if you like but it must at least start with 'shortcut:'.
2. The 'url' you post must be a shortcut url rather than an actual URL. It must contain a '%s' with a lowercase 's'. When you enter text into the textbox on the del.icio.us page the text will replace the '%s' after being percent-encoded. For example 'http://www.google.com/search?hl=en&q=%s' is the shortcut url for Google and if you type 'foo bar' into the textbox the URI you will navigate to would be 'http://www.google.com/search?hl=en&q=foo%20bar'.

Complaints
This is neat but I do have a few complaints:

1. The text from the textbox is percent-encoded before replacing the '%s'. Most sites use application/x-www-form-urlencoded which encodes spaces as '+' rather than '%20'.
2. The shortcut url format seems to be taken from Mozilla's Firefox Custom Keywords. Its a shame it wasn't based on something more adaptable like the OpenSearch URL template syntax.
3. A '%s' in the url means technically what you're submitting to del.icio.us isn't a URI as defined by the standard.
4. Allowing text after 'shortcut:' means you can't look at all of a user's shortcut using this tag.

The next step is to create a tool to sync my IE7 search providers with my shortcuts saved to delicious...technical howto tagging tool tag delicious