unix - Dave's Blog

Search

HTTP Compression Documentation Reference

2012 Jun 13, 3:08
There's a lot of name reuse in HTTP compression so I've made the following to help myself keep it straight.
HTTP Content Coding Token gzip deflate compress
An encoding format produced by the file compression program "gzip" (GNU zip) The "zlib" format as described in RFC 1950. The encoding format produced by the common UNIX file compression program "compress".
Data Format GZIP file format ZLIB Compressed Data Format The compress program's file format
Compression Method Deflate compression method LZW
Deflate consists of LZ77 and Huffman coding

Compress doesn't seem to be supported by popular current browsers, possibly due to its past with patents.

Deflate isn't done correctly all the time. Some servers would send the deflate data format instead of the zlib data format and at least some versions of Internet Explorer expect deflate data format instead of zlib data format.

PermalinkCommentscompress compression deflate gzip http http-header technical zlib

Time/Date Conversion Tool

2009 Aug 28, 3:39

I built timestamp.exe, a Windows command line tool to convert between computer and human readable date/time formats mostly for working on the first run wizard for IE8. We commonly write out our dates in binary form to the registry and in order to test and debug my work it became useful to be able to determine to what date the binary value of a FILETIME or SYSTEMTIME corresponded or to produce my own binary value of a FILETIME and insert it into the registry.

For instance, to convert to a binary value:

[PS C:\] timestamp -inString 2009/08/28:10:18 -outHexValue -convert filetime
2009/08/28:10:18 as FILETIME: 00 7c c8 d1 c8 27 ca 01

Converting in the other direction, if you don't know what format the bytes are in, just feed them in and timestamp will try all conversions and list only the valid ones:

[PS C:\] timestamp -inHexValue  "40 52 1c 3b"
40 52 1c 3b as FILETIME: 1601-01-01:00:01:39.171
40 52 1c 3b as Unix Time: 2001-06-05:03:30:08.000
40 52 1c 3b as DOS Time: 2009-08-28:10:18:00.000
(it also supports OLE Dates, and SYSTEMTIME which aren't listed there because the hex value isn't valid for those types). Or use the guess option to get timestamp's best guess:
[PS C:\] timestamp -inHexValue  "40 52 1c 3b" -convert guess
40 52 1c 3b as DOS Time: 2009-08-28:10:18:00.000

When I first wrote this I had a bug in my function that parses the date-time value string in which I could parse 2009-07-02:10:18 just fine, but I wouldn't be able to parse 2009-09-02:10:18 correctly. This was my code:

success = swscanf_s(timeString, L"%hi%*[\\/- ,]%hi%*[\\/- ,]%hi%*[\\/- ,Tt:.]%hi%*[:.]%hi%*[:.]%hi%*[:.]%hi", 
&systemTime->wYear,
&systemTime->wMonth,
&systemTime->wDay,
&systemTime->wHour,
&systemTime->wMinute,
&systemTime->wSecond,
&systemTime->wMilliseconds) > 1;

See the problem?

To convert between these various forms yourself read The Old New Thing date conversion article or Josh Poley's date time article. I previously wrote about date formats I like and dislike.

PermalinkCommentsdate date-time technical time windows tool

Command-line Fu - The best UNIX commands on the web

2009 Mar 23, 9:35Ohhh some nice ones in here. "Command-Line-Fu is the place to record those command-line gems that you return to again and again."PermalinkCommentsshell unix linux cli howto tips via:swannman

DoxPara Research

2008 Jul 14, 10:32Dan Kaminsky's blog.PermalinkCommentsdan-kaminsky blog security dns ip networking browser windows unix microsoft hack

URI Addressable Text Adventure Games

2008 Mar 2, 9:18

This post is about creating a server side z-code interpreter that represents game progress in the URI. Try it with the game Lost Pig.

I enjoy working on URIs and have the mug to prove it. Along those lines I've combined thoughts on URIs with interactive fiction. I have a limited amount of experience with Inform which generates Z-Code so I'll focus on pieces written in that. Of course we can already have URIs identifying the Z-Code files themselves, but I want URIs to identify my place in a piece of interactive fiction. The proper way to do this would be to give Z-Code its own mimetype and associate with that mimetype the format of a fragment that would contain the save state of user's interactive fiction session. A user would install a browser plugin that would generate URIs containing the appropriate fragment while you play the IF piece and be able to load URIs identifying Z-Code files and load the save state that appears in the fragment.

But all of that would be a lot of work, so I made a server side version that approximates this. On the Web Frotz Interpreter page, enter the URI of a Z-Code file to start a game. Enter your commands into the input text box at the bottom and you get a new URI after every command. For example, here's the beginning of Zork. I'm running a slightly modified version of the Unix version of Frotz. Baf's Guide to the IF Archive has lists of IF games to try out.

There are two issues with this thought, the first being the security issues with running arbitrary z-code and the second is the practical URI length limit of about 2K in IE. From the Z-Code standard and the Frotz source it looks like 'save' and 'restore' are the only commands that could do anything interesting outside of the Z-Code virtual machine. As for the length-limit on URIs I'm not sure that much can be done about that. I'm using a base64 encoded copy of the compressed input stream in the URI now. Switching to the actual save state might be smaller after enough user input.

PermalinkCommentszork frotz interactive-fiction zcode if technical uri fragment

Some Handy Commands - comm & seq

2008 Jan 13, 11:57Brief intro to comm & seq, two handy commands of which I was previously unaware.PermalinkCommentsvia:kris.kowal tool tools unix seq comm

DNA seen through the eyes of a coder

2008 Jan 3, 12:01The title says it. This is a description of the build process, make files, and programming language syntax that is life.PermalinkCommentsgeek humor genetics biology dna programming science evolution software unix

Date Time Formats

2007 Sep 27, 2:17Starting on a new simple project I wanted to get the history of my Delicious links. Delicious has an export tool available via the settings section so I thought I'd try that. However, the links aren't exported in XML not even in XHTML but rather in HTML. Shocking. An example:
"Don't Tase Me, Bro!" (UF Student Tasered Remix)
Remix of the 'Don't tase me, bro!' guy getting tasered.At this point I'm already not going to use this file because its in HTML but I'm even more disgusted by those date time values. Raymond Chen of the Old New Thing posted about recognizing timestamps and timestamp sentinel values. From the first blog post and with the use of a calculator for base conversion one can tell that those are UNIX style timestamps counting the number of seconds since 1970.

It reminds me of my hatred for the MIME date time format I developed working on my webpage's server side parsing of atom and RSS. Atom is of course my favorite as Atom uses the Internet date time format described in the following documents. Here's an example of one 2007-09-27T020:50:00.000-08:00 On the other hand the evil and villainous RSS uses the MIME date time format now described in the more recent IETF MIME standard. Here's an example Thu, 27 Sep 2007 20:50:00 -0800
The Internet date time format has the advantage of being so easy to sort. An alphabetic sort with normal C-style collation rules of strings containing Internet date times will also sort them chronologically. This is not the case for the MIME date time due to the preceding day of the week and the spelled out month name. This also means that when producing these you have to figure out the day of the week and when parsing them you have to match month names rather than just parsing out numbers. Anyway now days if I see mention of a date time in a new proposed standard or spec I be sure to point out the numerous advantages of the Internet date time format.
PermalinkCommentsdate xml html feed time technical date-time code atom rss

Welcome to Subsystem for UNIX-based Applications

2007 Aug 10, 9:25A POSIX subsystem to build UNIX applications on top of on Windows.PermalinkCommentsunix microsoft posix programming c windows via:swannman

Native Win32 ports of some GNU utilities

2007 Aug 9, 1:33Win32 versions of many common Unix commands.PermalinkCommentsunix linux tool tools download windows sourceforge shell software free

YouTube - Monzy performs at Stanford Univ.

2007 Apr 9, 5:00Monzy performing Kill-9 rapping about killing processes and generally insulting your ability to code, use Unix, etc.PermalinkCommentsvideo humor unix live rap music nerd nerd-core via:swannman

BashFaq - Greg's Wiki

2007 Mar 19, 10:54Tutorials and examples for making scripts for the Unix shell Bash.PermalinkCommentshowto linux bash reference shell script tutorial programming tips

Xming Notes

2007 Jan 11, 12:01Xmin lets you run your X Windows applications remotely via SSH on Windows. Useful with my new server. Awesome!PermalinkCommentswindows x11 xserver software linux x xwindows client free open-source opensource remote unix

Reflections on Trusting Trust

2005 Dec 29, 3:09PermalinkCommentssecurity unix history development hack acm ken-thompson trust

Sed - An Introduction and Tutorial

2005 Dec 6, 11:50PermalinkCommentsdevelopment reference regex tools unix sed tutorial

Gentoo Linux Documentation -- Learning vi -- the "cheatsheet" technique

2005 Apr 10, 1:27PermalinkCommentslinux vim gentoo unix reference

Odd Comments and Strange Doings in Unix

2005 Mar 28, 10:26PermalinkCommentshumor unix development
Older Entries Creative Commons License Some rights reserved.