mit page 7 - Dave's Blog

Backup Notes

2007 Jul 13, 8:30I bought an external backup drive a few weekends ago. I've previously setup a Subversion repository so I decided to move everything into the repository and then back it up. So in went the contents of all of my %USERPROFILE% and ~ directories with a bit of sorting and pruning. Not too much though given its much easier to dump in everything and search for what I want then to take the time to examine and grade each file. What follows are the notes I took while setting this up. It takes me a bit of time to look up the help on each command so I figure I'll write it all down here for the benefit of myself and potentially others...

Setting Up the Backup Drive For Linux
I first changed the filesystem on the drive to ext3. I plugged it into my USB2.0 port and ran fdisk:

sudo fdisk /dev/sda

Useful commands I used to do this follow mostly in order:

m

help

p

print current partitions

d

delete current partition

n

create new partition (I used the defaults)

w

write changes and exit

Then I formatted for ext3.

sudo mkfs.ext3 /dev/sda1

I made it easy to mount:

sudo vim /etc/fstab
# added line to end:
/dev/sda1 /media/backup ext3 rw,user,noauto 0 0

I setup the directory structure on the disk

mount /media/backup
sudo mkdir /media/backup/users
sudo mkdir /media/backup/users/dave
sudo chown dave:dave /media/backup/users/dave

After all that its easy to make a copy of the Subversion repository:

mount /media/backup
cp -Rv /home/dave/svn /media/backup/users/dave/
umount /media/backup

Next on the agenda is to add a cron job to do this regularly.

Subversion Command Reference
On a machine that has local access to the Subversion repository you can check out a specific subdirectory as follows using the file scheme:

svn co file:///home/dave/svn/trunk/web/dave%40deletethis.net/public_html

Note also that although one of my directories is named 'dave@deletethis.net' Subversion requires the '@' to be percent-encoded.
Other useful subversion commands:

svn help

help

svn list file:///home/dave/svn/

list all files in root dir of svn depot

svn list -R file:///home/dave/svn/

list all files in svn depot

svn list -R file:///home/dave/svn/ | grep \/$

list all directories

svn status

List status of all files in the working copy directory as in - modified, not in repository, etc

svn update

Brings the working copy up to date wrt the repository

svn commit

Commit changes from the working copy to the repository

svn add / move / delete

Perform the specified action -- occurs immediately

Setting up Windows Client for Auto Auth into SVN
When using an SVN client on Windows via svn+ssh its useful to have the Windows automatically generate connections to the SVN server. I use putty on my Windows machines so I read the directions on using public keys with putty.

putty.exe dave@deletethis.net
cd .ssh
vim authorized_keys # leave the putty window open for now
puttygen.exe
Click the 'generate' button
Move the mouse around until finished
Copy text in 'Public key for pasting into OpenSSH authorized_keys file:' to putty window & save & close putty window
Enter Key passphrase & Comment in puttygen
Save the private key somewhere private
pageant.exe
'Add Key' the private key just saved.

Checking out using Tortoise SVN
On one of my Windows machines I've already installed Tortoise SVN. Checking out from my SVN repository was really easy. I just right clicked in Explorer in a directory and selected "SVN Checkout...". Then in the following dialog I entered the svn URI:

svn+ssh://dave@deletethis.net/home/dave/svn/trunk/web/dave%40deletethis.net/public_html/

Note again that the '@' that is part of the directory name is percent-encoded as '%40' while the '@' in the userinfo is not.

Windows Command Line Check Out
On my media center I didn't want to install Tortoise SVN so rather I used the command line tool. I setup pageant like before the only difficulty was getting the SVN command line tool to use putty. With the default configuration you can use the SVN_SSH environment variable to point at a compliant SSH command line tool. The trick is that its interpreted as a backslash escaped string. So I set mine thusly:

set SVN_SSH=C:\\users\\dave\\bin\\putty\\plink.exe

The escaping solved the vague error I received about not being able to create the tunnel.backup technical personal windows svn linux subversion

Second Life Translator

2007 Jul 4, 10:58Hackdiary
I really enjoy reading Matt Biddulph's blog hackdiary. An entry some time ago talked about his Second Life flickr screen which is a screen in Second Life that displays images from flickr.com based on viewers suggested tags. I'm a novice to the Second Life scripting API and so it was from this blog post I became aware of the llHTTPRequest. This is like the XMLHttpRequest for Second Life code in that it lets you make HTTP requests. I decided that I too could do something cool with this.

Translator
I decided to make a translator object that a Second Life user would wear that would translate anything said near them. The details aren't too surprising: The translator object keeps an owner modifiable list of translation instructions each consisting of who to listen to, the language they speak, who to tell the translation to, and into what language to translate. When the translator hears someone, it runs through its list of translation instructions and when it finds a match for the speaker uses the llHTTPRequest to send off what was said to Google translate. When the result comes back the translator simply says the response.

Issues
Unfortunately, the llHTTPRequest limits the response size to 2K and no translation site I can find has the translated text in the first 2K. There's a flag HTTP_BODY_MAXLENGTH provided but it defaults to 2K and you can't change its value. So I decided to setup a PHP script on my site to act as a translating proxy and parse the translated text out of the HTML response from Google translate. Through experimentation I found that their site can take parameters text and langpair queries in the query like so: http://translate.google.com/translate_t?text=car%20moi%20m%C3%AAme%20j%27en%20rit&langpair=fr|en. On the topic of non US-ASCII characters (which is important for a translator) I found that llHTTPRequest encodes non US-ASCII characters as percent-encoded UTF-8 when constructing the request URI. However, when Google translate takes parameters off the URI it only seems to interpret it as percent-encoded UTF-8 when the user-agent is IE's. So after changing my PHP script to use IE7's user-agent non US-ASCII character input worked.

In Use
Actually using it in practice is rather difficult. Between typos, slang, abbreviations, and the current state of the free online translators its very difficult to carry on a conversation. Additionally, I don't really like talking to random people on Second Life anyway. So... not too useful.personal translate second-life technical translator sl code google php llhttprequest

IE7 Feed Display Update

2007 May 22, 3:22I've created an update to the IE7 feed display.

After working on my update to the XML source view I tried running my resourcelist program on other IE DLLs including ieframe. I found that one of the resources in ieframe is the XSLT used to turn an RSS feed into the IE7 feed display.

My first thought for this was that I could embed enclosures into the feed display. For instance, have controls for youtube.com videos or podcast audio files directly in the feed display. However, I found that I can't use object or embed tags that rely on ActiveX controls in the page or in frames in the feed display.

With that through I decided I could at least add support for some RSS extensions. Thanks to IE7's RSS platform which provides a normalized view of RSS feeds it was really easy to do this. I went to several popular RSS feeds and RSS feeds that I like and took a look at the source to see what extensions I might want to add support for.

For digg.com I added support for their RSS extension which includes digg count, and submitter name and icon. I added the digg count in a box on the right and tried to make it fit in stylistically. For the iTunes RSS extension I add the feed icon, feed author, and descriptions. I was surprised by how much of the podcasts content was missing from the feed view. I also added support for a few other misc things: the slash RSS extension's section and department, the feed description to the top of the feed display, and the atom author icon.

I wonder what other goodies lurk in IE's resources...feed res slashdot digg resource itunes technical browser ie rss extension

XPointer Framework - IE7 XML Source View Upgrade Part 3

2007 May 17, 5:16Previously I created some resource tools and then I used them to overwrite msxml3's XML source view. In this update I've added support for the XPointer Framework.

This time around I've started to add support for the XPointer Framework to my XML source view and I've added installation instructions. The framework consists of a series of pointer segments each of which has a scheme name followed by data in parenthesis. For example 'scheme1(data1)scheme2(data2)scheme3(data3)'. A pointer segment resolves to a portion of the XML document based on the data and the scheme name. The whole pointer resolves to the first segment that successfully resolves. That is, from the example, if scheme1 resolves to nothing and scheme2 resolves to something then that's used and scheme3 is ignored. In addition to the framework I've added support for the xmlns scheme which binds namespace prefixes to a namespace URI and the element scheme which is a simple way to resolve to particular elements in an XML. I also have limited support for the xpointer scheme the content of which is resolved as an XPath with some extra functions (which I don't support -- hence the limited). I've also thrown in schemes for the two SelectionLanguage values supported by msxml3.

Next time I might try to support the xpointer functions that aren't in xpath using msxml script. But I think I'm losing steam on this project... we'll see.resource technical xml xpointer res xpath xslt

New XSLT - IE7 XML Source View Upgrade Part 2

2007 May 11, 8:55Last time, I had written some resource tools to allow me to view and modify Windows module resources in my ultimate and noble quest to implement the XML content-type fragment in IE7. Using the resource tools I found that MSXML3.DLL isn't signed and that I can replace the XSLT embedded resource with my own, which is great news and means I could continue in my endevour. In the following I discuss how I came up with this replacement for IE7's XML source view.

At first I thought I could just modify the existing XSLT but it turns out that it isn't exactly an XSLT, rather its an IE5 XSL. I tried using the XSL to XSLT converter linked to on MSDN, however the resulting document still requires manual modification. But I didn't want to muck about in their weird language and I figured I could write my own XSLT faster than I could figure out how theirs worked.

I began work on the new XSLT and found it relatively easy to produce. First I got indenting working with all the XML nodes represented appropriately and different CSS classes attached to them to make it easy to do syntax highlighting. Next I added in some javascript to allow for closing and opening of elements. At this point my XSLT had the same features as the original XSL.

Next was the XML mimetype fragment which uses XPointer, a framework around various different schemes for naming parts of an XML document. I focused on the XPointer scheme which is an extended version of XPath. So I named my first task as getting XPaths working. Thankfully javascript running in the HTML document produced by running my XSLT on an XML document has access to the original XML document object via the document.XMLDocument property. From this this I can execute XPaths, however there's no builtin way to map from the XML nodes selected by the XPath to the HTML elements that I produced to represent them. So I created a recursive javascript function and XSLT named-template that both produce the same unique strings based on an XML node's position in the document. For instance 'a3-e2-e' is the name produced for the 3rd attribute of the second element of the root element of the XML document. When producing the HTML for an XML node, I add an 'id' attribute to the HTML with the unique string of the XML node. Then in javascript when I execute an XPath I can discover the unique string of each node in the selected set and map each of them to their corresponding positions in the HTML.

With the hard part out of the way I changed the onload to get the fragment of the URI of the current document, interpret it as an XPath and highlight and navigate to the selected nodes. I also added an interactive floating bar from which you can enter your own XPaths and do the same. On a related note, I found that when accessing XML files via the file URI scheme the fragment is stripped off and not available to the javascript.

The next steps are of course to actually implement XPointer framework parsing as well as the limited number of schemes that the XPointer framework specifies.xml xpointer msxml res xpath xslt resource ie7 technical browser ie xsl

Opera Speed Dial Code-less Hack for IE7

2007 Apr 17, 11:45Opera (the fifth most popular web browser) has a new feature named Speed Dial (video of it in action). Whenever you open a new tab you get your Speed Dial view which consists of nine thumbnails of user-settable pages. Its like a quick-favorites that appears every time you open a new tab. I think this is a neat idea and was considering how I might do that in IE7. The following is my hack-y and ugly but no coding required version of Speed Dial for IE7. I like my hack and I'm about to expound upon it in unnecessary detail so skip to the last paragraph if you're afraid of losing interest.

By default in IE7, whenever you open a new tab you navigate to 'about:Tabs'. As noted in wikipedia the result of navigation to 'about:Tabs' is determined by values in the registry. Specifically, values in the key in "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\AboutURLs". Usually this fact is exploited by malicious software to hijack "about:blank" and show you ads but we can hijack it too in order to display our Speed Dial-ish page.

Of course since this is a code-less hack we've got limited options on what to change 'about:Tabs' to display. It should have the following requirements.

1. Something local so that our 'about:Tabs' doesn't disappear when we go offline and so that its relatively fast.
2. The user should be able to modify its content.
3. Show links that the user uses.
4. Show thumbnails of those links
5. Provide easy to use drag and drop interaction and generally look cool.

Now, I use del.icio.us which allows me to store all of my favorites online and which provides RSS feeds that list my saved links. New in IE7 is an RSS platform that will, among other things, cache RSS feeds locally. So, by pointing about:Tabs to my del.icio.us feed 'http://del.icio.us/rss/sequelguy/quickreference' I get (1) from IE7's RSS support, and (2) and (3) from del.icio.us. Of course requirements (4) and (5) are missing but hey, I said this was ugly.

In summary, if you change the registry value "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\AboutURLs!Tabs" to point to an RSS feed of your favorites you can get a hack-y version of Opera's Speed Dial. I should note that although its referenced on pages such as wikipedia changing your 'about:Tabs' URI in the manner I describe is not documented and not supported by Microsoft. There could be all kinds of horrible repercussions from this change of which I'm not aware. Yeah, actually you know what? Forget I said any of this. Pretend I never wrote it...browser technical hack

Delicious shortcut tag

2007 Apr 8, 3:05Shortcut Tag?
I just saw this on another user's delicious links: a link to ESV search that's tagged with, among other things, "shortcut:esv". When viewed on del.icio.us there's a text box that lets you search using that link. I hadn't seen this before, but it seems pretty cool and I'm surprised I hadn't seen it previously. A delicious post with such a tag ends up looking like the following: I tried searching for information on this and I've found other delicious users doing the same thing, but nothing about the tag itself. If you know any information especially official information from del.icio.us itself please post links in reply to this post. So without further preface here's what I've learned about the del.icio.us shortcut tag.

How-to
To get a search box in your del.icio.us links make a post that satisfies the following requirements:

1. One of the tags must begin with the text 'shortcut:'. You can have more text following that in the tag if you like but it must at least start with 'shortcut:'.
2. The 'url' you post must be a shortcut url rather than an actual URL. It must contain a '%s' with a lowercase 's'. When you enter text into the textbox on the del.icio.us page the text will replace the '%s' after being percent-encoded. For example 'http://www.google.com/search?hl=en&q=%s' is the shortcut url for Google and if you type 'foo bar' into the textbox the URI you will navigate to would be 'http://www.google.com/search?hl=en&q=foo%20bar'.

Complaints
This is neat but I do have a few complaints:

1. The text from the textbox is percent-encoded before replacing the '%s'. Most sites use application/x-www-form-urlencoded which encodes spaces as '+' rather than '%20'.
2. The shortcut url format seems to be taken from Mozilla's Firefox Custom Keywords. Its a shame it wasn't based on something more adaptable like the OpenSearch URL template syntax.
3. A '%s' in the url means technically what you're submitting to del.icio.us isn't a URI as defined by the standard.
4. Allowing text after 'shortcut:' means you can't look at all of a user's shortcut using this tag.

The next step is to create a tool to sync my IE7 search providers with my shortcuts saved to delicious...technical howto tagging tool tag delicious

OpenID Stolen Thoughts

2007 Mar 13, 7:57I had a few thoughts after reading about OpenID. However, after doing only a very small amount of digging I can see these aren't new thoughts.

Anonymous OpenID

Have an OpenID that anyone can use because it performs no authorization. You'd specify a URI like http://deletethis.net/anonymousopenid/yournamehere and you'd immediately get an anonymous OpenID associated with that URI. This has already been implemented by Jayant Gandhi.

Group OpenID

Have an OpenID that consists of a group of member OpenIDs. To login as the Group OpenID you need to login with any of the member OpenIDs. This is discussed more by Dmitry Shechtman on his blog.

OpenID Normalization

I find that I already have a couple of OpenIDs without even trying due to AOL giving out OpenIDs. I'd like for all of my OpenIDs to point to one canonical OpenID. It looks like this may already be possible by the OpenID specification.

I guess I'm a little late to the scene.technical stolen-thoughts openid

Encode-O-Matic Update

2006 Dec 3, 12:28I've updated Encode-O-Matic again. This is a tool I'm working on to convert between various Internet related encodings such as character sets, HTML encoding, URI encoding, base64, and IDN. In this update I've put it all into an installer. I'm using Nullsoft's installer generator to produce the installer. I've added a Base Conversion converter to convert between arbitrary bases and a Reverse converter that reverses the input by character, byte, or strings with arbitrary delimiters.installer encodeomatic project charset nullsoft encoding