deletethis.net

Edge and WebView2 in Chrome DevTools MCP

2026-03-19T00:00:00Z

The Chrome DevTools MCP is very useful! Although it is the Chrome DevTools MCP, it supports connecting to pretty much all chromium based browsers. This includes Microsoft Edge and WebView2 with a bit of extra configuration in mcp.json. The tool knows explicitly about Chrome, so you have to specify either the --executablePath for launch scenarios or --userDataDir for auto-connect scenarios.

Edge Launch #

To launch Edge you set the executablePath to point to your Edge binary. For example the following uses the Edge Dev channel on Windows:

"args": [
  "--registry",
  "https://registry.npmjs.org",
  "chrome-devtools-mcp@0.18.1",
  // Relevant part next:
  "--executablePath=C:\\Program Files (x86)\\Microsoft\\Edge Dev\\Application\\msedge.exe"
]

There are a few different locations where Edge could be installed so you'll need to ensure you have the correct one for your device.

Default Edge install paths (not definitive, may vary based on user configuration, version or group policies):

OS	Channel	Path
Windows	Stable	`%PROGRAMFILES(X86)%\Microsoft\Edge\Application\msedge.exe`
Windows	Beta	`%PROGRAMFILES(X86)%\Microsoft\Edge Beta\Application\msedge.exe`
Windows	Dev	`%PROGRAMFILES(X86)%\Microsoft\Edge Dev\Application\msedge.exe`
Windows	Canary	`%LOCALAPPDATA%\Microsoft\Edge SxS\Application\msedge.exe`
macOS	Stable	`/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge`
macOS	Beta	`/Applications/Microsoft Edge Beta.app/Contents/MacOS/Microsoft Edge`
macOS	Dev	`/Applications/Microsoft Edge Dev.app/Contents/MacOS/Microsoft Edge`
macOS	Canary	`/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge`
Linux	Stable	`/usr/bin/microsoft-edge`
Linux	Beta	`/usr/bin/microsoft-edge-beta`
Linux	Dev	`/usr/bin/microsoft-edge-dev`
Linux	Canary	Not available

Edge Auto-connect #

To auto-connect to Edge, you need to:

Setup Edge for remote debugging:
1. Either use the --remote-debugging-port=0 flag when launching Edge
2. Or start Edge, navigate to edge://inspect/#remote-debugging, and enable remote debugging on that page
And then set the --user-data-dir to point to the Edge user data directory in mcp.json:

"args": [
  "--registry",
  "https://registry.npmjs.org",
  "chrome-devtools-mcp@0.18.1",
  // Relevant part next:
  "--autoConnect",
  "--user-data-dir=C:\\Users\\davris\\AppData\\Local\\Microsoft\\Edge Dev\\User Data"
]

Default Edge user data directories (not definitive, may vary based on user configuration, version or group policies):

OS	Channel	Path
Windows	Stable	`%LOCALAPPDATA%\Microsoft\Edge\User Data`
Windows	Beta	`%LOCALAPPDATA%\Microsoft\Edge Beta\User Data`
Windows	Dev	`%LOCALAPPDATA%\Microsoft\Edge Dev\User Data`
Windows	Canary	`%LOCALAPPDATA%\Microsoft\Edge SxS\User Data`
macOS	Stable	`~/Library/Application Support/Microsoft Edge`
macOS	Beta	`~/Library/Application Support/Microsoft Edge Beta`
macOS	Dev	`~/Library/Application Support/Microsoft Edge Dev`
macOS	Canary	`~/Library/Application Support/Microsoft Edge Canary`
Linux	Stable	`~/.config/microsoft-edge`
Linux	Beta	`~/.config/microsoft-edge-beta`
Linux	Dev	`~/.config/microsoft-edge-dev`
Linux	Canary	Not available

WebView2 Auto-connect #

WebView2 is similar to the Edge auto-connect scenario. WebView2 doesn't have a launch scenario because its up to the host app to decide when to create a WebView2.

To auto-connect to WebView2, you need to:

Setup WebView2 for remote debugging using WebView2 loader overrides:
1. Either use WebView2Utilities and follow the How to: Auto open DevTools wiki page but instead of checking the Auto open DevTools checkbox, type --remote-debugging-port=0 in the Arguments textbox.
2. Or manually create a registry value to set the AdditionalBrowserArguments after replacing appname.exe with the name of your WebView2 host executable.

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Edge\WebView2\AdditionalBrowserArguments]
"appname.exe"="--remote-debugging-port=0"

And then set the --user-data-dir to point to the WebView2 user data directory in mcp.json. You'll need to discover the user data directory of the host app. You can use WebView2Utilities to do this as well, by going to the Host Apps tab, selecting your running host app, and looking at the User data folder row. For WebView2 user data folders, the path should end with EBWebView. This is automatically added by WebView2 so if you are copying the path from source code, you will need to add it yourself.

"args": [
    "--registry",
    "https://registry.npmjs.org",
    "chrome-devtools-mcp@0.18.1",
    // Relevant part next:
    "--autoConnect",
    "--user-data-dir=C:\\Users\\davris\\AppData\\Local\\Packages\\Microsoft.WinUI3ControlsGallery_8wekyb3d8bbwe\\LocalState\\EBWebView"
],

WebView2 wv2winrt bug case study

2026-03-05T00:00:00Z

The following are my notes on investigating a bug that a customer ran into back when I was on the WebView2 team. I'm sharing it here in case aspects of it can be helpful for folks in similar situations. The names of files, functions, and such have been changed. The customer had provided me a VS solution that would reproduce the bug somewhat minimally:

I've started the app, opened DevTools for WebView2, turned on pause on exceptions, then I reproduced the issue by clicking the appropriate button. JavaScript throws an exception and DevTools breaks in where we can see its a stack in our AddHostObjectToScript JavaScript code handling an incoming EmbeddedBrowserWebViewEvent message. To elaborate, part of WebView2.AddHostObjectToScript is implemented using JavaScript that runs in the context of the target HTML document. That JavaScript is responsible for producing JavaScript proxy objects that represent its corresponding native host object, proxying method calls and their results, property gets and sets, and so on. Looking at the exception information in the data JSON property on the message object in the DevTools we can see it is telling us there was an error Member not found. (0x80020003).

This is a response message from the native side to a request message that the JavaScript side would have initiated. The common issue for this error means that the JavaScript side asked to run a method that doesn't exist. Though in this case the customer says something else is going on.

Next I look at the JavaScript to see what the corresponding request was. Its something in here so I rerun the repro but with a breakpoint on the start of this function to see when we hit the error (though I know it has to be the await call because the response we saw coming in was async and everything else in here is sync):

        async function exampleItemsAsync() {
            try {
                const instance = new Demo.Example();
                var result = await instance.getExampleItemsAsync();
                for (var v of result) {
                    logMessage('Enumerated Result Async: ' + v.identifier);
                }
            } catch (err) {
                logMessage('Async Error: ' + err);
            }
        }

I can see it is the await call.

At this point I might normally check that the GetExampleItemsAsync method exists in the C# class, in the winmd, in the wv2winrt tool output saying it found it, and in the generated wv2winrt C++ source. I would check the wv2winrt tool generated code under Project\WinRTAdapter\Generated Files\wv2winrt\Demo.g.cpp to see that there is code generated for the function, I would open Project\x64\Debug\WinRTAdapter\WinRTAdapter.winmd in ILSpy and see that it contains the method. But the customer says that's all there and I know this customer is good and has done their work. So I try putting a breakpoint in Demo.Example.GetExampleItemsAsync() and when I run the repro I can see that code is getting called.

So it seems like there's something wrong happening between calling GetExampleItemsAsync and the JavaScript projection of the result. We saw the JavaScript projection receiving the message from native and it was already an error at that point and I stepped through and saw the end of the GetExampleItemsAsync() method succeeded, so its somewhere in between. Our options for code in between is the AddHostObjectToScript C++ WebView2 code, the AddHostObjectToScript C# code (which shouldn't be involved in this case if this is truly just WinRT) and the wv2winrt generated C++ code.

The wv2winrt generated code is under here: Project\WinRTAdapter\Generated Files\wv2winrt\Demo.g.cpp and where I'm guessing the issue lies. I open that file, search for GetExampleItemsAsync, and put some breakpoints in at invocation (its marked with a comment saying its for // Invoke for ...) and in the async completed handler (the callback lambda passed as a parameter to the asyncResultAsWinRT.Completed method call). Note I had to turn on mixed mode debugging in the VS project settings.

Then running the repro I see the invocation breakpoint hit, the C# code breakpoint I set earlier hits, then the async completed handler, then an exception is getting thrown from the c++/winrt code during our generated code call to Convert:

resultAsVariant.reset(Converter<decltype(resultAsWinRT), VARIANT>(
    dispatchAdapter).Convert(resultAsWinRT));

This generated code is the boundary between the WinRT code and the COM IDispatch code that AddHostObjectToScript deals with. This line is trying to convert the WinRT type we got as a result of the async call, into a COM IDispatch related type that AddHostObjectToScript can deal with.

Looking at the method in our stack just before the winrt::check_hresult call we can see the failure is coming from the following:

HRESULT hr = CreateDispatchFromInspectable(
    unwrappedAsABI.get(), dispatchAdapter, dispatch.put(), &is_cacheable);

I put a breakpoint on this line but disable it because it will get hit a bunch during the repro. I rerun the repro, and when the async completed handler bp hits, I reenable this bp. Stepping into this function the first interesting thing to check is the IInspectable.RuntimeClassName which is the name of the WinRT type we're trying to convert to. In this case its "Windows.Foundation.Collections.IVector`1<Demo.ExampleItem>" which is a WinRT type for a vector of ExampleItems. Stepping through the code the call to LookupInstanceConstructibleEntry returns nullptr.

LookupInstanceConstructibleEntry should return a wrapper class we can use to wrap an IInspectable (WinRT type) of the specified type name. This list and the wrapper classes are all generated in the wv2winrt generated code. If the IVector\1` typename isn't generated then there may be either something wrong with wv2winrt that it doesn't generate this, or we have to adjust the wv2winrt configuration to tell it to include this type.

Just to make sure I understand what code was generated I search for IVector in the wv2winrt code. In globals.cpp we can see the s_instanceConstructibleEntries array that LookupInstanceConstructibleEntry was checking. It is sorted alphabetically and in it we see some entries for IVector but not the type we want. So it looks like wv2winrt tool didn't include this type. We can look at the wv2winrt tool output from when WinRTAdapter is built to see if there's any clues. With verbose output turned on wv2winrt tool will list why it is including or excluding types. After rebuilding WinRTAdapter I search for IVector in the tool's output:

1>Including via reference: Demo.Example.GetListOfOther -> Windows.Foundation.Collections.IVector`1<Demo.MyClass>
1>Including via reference: Demo.Example.GetExampleItemsAsync -> Windows.Foundation.IAsyncOperation`1<Windows.Foundation.Collections.IIterable`1<Demo.ExampleItem>>

Conveniently, GetListOfOther gives us an example of something that is working and producing a found IVector type. The main differences are that GetListOfOther is not async and it is also listed here as returning an IVector whereas GetExampleItemsAsync is async and returning an IIterable.

Lower in the tool output we see it finding the async's completed type:

1>Including via reference: Windows.Foundation.IAsyncOperation`1<Windows.Foundation.Collections.IIterable`1<Demo.ExampleItem>>.GetResults -> Windows.Foundation.Collections.IIterable`1<Demo.ExampleItem>

So it doesn't seem like the issue is that its async. Opening Project\x64\Debug\Demo.winmd in ILSpy I can see that the winmd does indeed seem to say IAsyncOperation<IIterable<ExampleItem>> IExampleClass.GetExampleItemsAsync();

It seems to be the issue since at runtime the typename for the returned object was IVector<ExampleItem> not IIterable<ExampleItem>.

To resolve this we should either get C#/WinMD tool to return IVector instead of IIterable, or get the wv2winrt tool to generate code for IVector<ExampleItem>. I'm going to try the second option.

I open the project file (Project\WinRTAdapter\WinRTAdapter.vcxproj) in VSCode and examine the WebView2* properties directly:

<WebView2DispatchAdapterIncludeFilters>Demo;Windows.System.UserProfile;Windows.Globalization.Language;$(WebView2DispatchAdapterIncludeFilters)</WebView2DispatchAdapterIncludeFilters>
<WV2WinRTVerbosity>high</WV2WinRTVerbosity>

(Notice that WV2WinRTVerbosity is already set to 'high' - this is important in order to get the wv2winrt tool verbose output that helped us earlier)

The WebView2DispatchAdapterIncludeFilters property is a bunch of namespace or type names that should be included by the wv2winrt code generation. We can try adding our type name here. When I do this I start getting strange errors. I expect this is a problem with the \`` and <>` characters and wv2winrt tool not handling it properly or the command line not getting constructed with them correctly. This is likely a wv2winrt bug although perhaps I'm messing up their encoding in the project file.

Instead I try the first solution, I add the following code to Example2.cs:

public IAsyncOperation<IList<ExampleItem>> DummyMethod() { return null; }

Now rebuilding Demo and WinRTAdapter everything is building. Looking in the wv2winrt output from WinRTAdapter we can see the following:

1>Including via reference: Demo.Example.DummyMethod -> Windows.Foundation.IAsyncOperation`1<Windows.Foundation.Collections.IVector`1<Demo.ExampleItem>>

And now running the bug repro the issue no longer occurs.

Having this dummy method is a workaround to the issue. A better solution would be to add the IVector\1` type to the project's WebView2DispatchAdapterIncludeFilters.

The real issue is that the runtime object and the winmd disagree about the type that's getting returned IIterable vs IVector which should be caused by CsWinRT the tool that is connecting up the C# to WinRT.

Code Search Comparison: GitHub, Azure DevOps, and Chromium

2025-08-06T00:00:00Z

Before working in Chromium I had been using our internal project's Azure DevOps (ADO) code search feature, and occassionally using Chromium Code Search or GitHub's code search. It seemed like Chromium Code Search or GitHub code search would only sometimes work with my queries. Sitting down to compare the syntax of them led me to figure out why and the power of Chromium Code Search:

Comparison Table #

Feature	ADO	Chromium	GitHub
Wildcard syntax	Full glob match (`*`)	Partial regex match (uses regular expressions, e.g. `.*`)	Paths use partial glob, everything else is partial literal match, or partial regex with slashes
Search for method/class definition/declaration	`Def:Name` / `decl:Name`	`Class:Name` / `Func:Name`	`Symbol:Name`
Negation logic operator	`NOT`	`-`	`NOT`
Includes generated files	❌	✅	❌
Documentation	Functional code search - Azure Repos \| Microsoft Learn	Syntax reference \| Code Search \| Google for Developers	Understanding GitHub Code Search syntax

The biggest difference is learning that Chromium is using partial regular expressions matching, ADO is using full glob matching, and GitHub is using partial literal matching unless you use /.../ to explicitly ask for regular expressions. So TLDR: * in ADO should be .* in Chromium, and /.*/ in GitHub.

Example Comparison #

Searching for the kTopLevel symbol in the Chromium codebase, excluding tests and mojom files, and only looking in a service path:

ADO query:

kTopLevel NOT path:*test* NOT ext:mojom path:*service*

cs.chromium query:

\bkTopLevel\b -path:test -path:\.mojom$ -path:^out/ path:service

GitHub query:

repo:chromium/chromium /\bkTopLevel\b/ NOT path:test NOT path:*.mojom PATH:service

Notable Differences: #

`kTopLevel` #

ADO: kTopLevel matches just that word unless you use * to match more.
cs.chromium: kTopLevel is a regular expression that matches parts of words and you need to use \b on the ends to indicate that it shouldn't match more.
GitHub: kTopLevel on its own does a partial literal match. If you want to match the whole word you can use a regular expression with /.../ around it, so /\bkTopLevel\b/.

`NOT path:test` #

ADO: We use NOT for negation and to match any path with test we again have to use * to match more
cs.chromium: We negate with – and test is a regular expression that matches the text test anywhere in the path.
GitHub: We negate with NOT and test is a partial glob matches the text test anywhere in the path.

`NOT ext:mojom` #

ADO: We can use ext: to match file extensions. It's just a shortcut for saying path:*.mojom.
cs.chromium: We need to represent this as a regular expression -path:\.mojom$. In this case escape the . with \ so it's a literal . and use $ to indicate that this is the end of the path.
GitHub: We can use path:*.mojom to match the file extension, although its a partial glob match so it would also match something like my.mojom.txt and is not perfect.

`-path:^out/` #

cs.chromium: They index generated files. Depending on what you're looking for this can be incredibly useful or since it indexes multiple different archs and platforms of generated code it can produce a lot of noise. You can use -path:^out/ to ensure no generated files under the gen build folder show up.

WatchBuildLog VS Code Extension

2025-07-20T00:00:00Z

I've made WatchBuildLog, a VS Code extension that watches a build log file and parses out build errors for the VS Code Problems tab. The default build log paths and build error patterns match the Chromium build system. But you can change the log paths and error patterns in the settings.

There are other VS Code extensions for running the build from VS Code. But Chromium has so many different build targets and build options, that I want to continue building like I normally do from the command line. So rather than starting the build from VS Code, I stay in the command line unless there are errors I want to fix in VS Code, then I switch to VS Code, and the build errors are waiting for me.

Development #

This is my first VS Code extension. I used GitHub Copilot in part of its creation and it was a reasonable experience. I did end up rewriting most of the code as the generated code was unnecessarily complicated and missed edge cases. But having never made a VS Code extension, before it was convenient to have Copilot help me get started and get the basic structure of the extension.

Time Travel Debugging WPT on Windows

2025-07-17T00:00:00Z

I've had trouble getting Time Travel Debugging (TTD) working with debugging WPT from my chromium repo on Windows. Having finally gotten it working, I wanted to document the steps I took to get it working:

Follow the install TTD instructions. This will install the Microsoft.TimeTravelDebugging app package and will expose the ttd.exe command line executable.
Disable hardware enforced stack protection. See Kernel-mode Hardware-enforced Stack Protection of where to find the option, but disable it. TTD cannot trace an executable with this enabled.
Add a WPT virtual test suite to pass in --no-sandbox. TTD doesn't work when Chromium sandboxing is on for a process. Edit third_party/blink/web_tests/VirtualTestSuites and add a new entry like the following:

  {
    "prefix": "no-sandbox",
    "platforms": [
      "Linux",
      "Mac",
      "Win"
    ],
    "bases": [ "" ],
    "args": [
      "--no-sandbox"
    ],
    "owners": [
      "example@example.com"
    ],
    "expires": "Jul 17, 2026"
  },

Run TTD to monitor the WPT headless_shell.exe processes. In an admin prompt run the following, but modify the -out path to wherever you want the TTD trace files to be created. Pick a path that can store files of several GBs in size. Leave this running.

Admin> ttd.exe -out C:\users\davris\tmp -monitor headless_shell.exe -noUI

Run your WPT tests using the virtual test suite, with timeouts extended, and using a release build in a non-admin prompt. Note in the following command the test starts with the test suite prefix virtual/no-sandbox/ and the --timeout-multiplier is set to 5 to allow for the much slower execution when TTD tracing.

Non-Admin> Q:\cr\src\third_party\blink\tools\run_wpt_tests -t Q:\cr\src\out\release_x64 virtual/no-sandbox/external/wpt/service-workers/service-worker/service-worker-header.https.html --no-retry-failures --verbose --driver-logging --timeout-multiplier 5

Once the WPT finishes you should find a bunch of *.run files in the output directory you specified in step 4. These are the TTD trace files. You can open them in WinDbg and use !peb to see the command line and which kind of chromium process kind. But usually the .run file of the renderer process you want is the last one and largest one, and for the browser process is the first one.

OpenSourceIn Browser Extension

2025-07-16T00:00:00Z

I've been writing (and using) a browser extension called OpenSourceIn that allows you to open the source code you're viewing in the browser into VS Code. It supports the web sites I use: GitHub, Azure DevOps, and the various Chromium development sites for bugs, code review, and source search. Working on Chromium I find myself very regularly switching between cs.chromium.org, issues.chromium.org, and chromium-review.googlesource.com and just wanted something to make it slightly easier to switch from the browser into the corresponding local source file. The best solution to this would be proper integration of these various sites into VS Code, but this provides good value for very little investment.

Development #

I used GitHub Copilot throughout the development of this extension and overall I'm not sure if it saved me time but probably helped me actually finish. My overall thoughts:

Side project fatigue mitigation #

I often get very interested and into a side project idea, but at this point in my life I can tell how many days I have before my focus and interest fails. GitHub Copilot helps me get through things like setting up the project, getting test infra setup, and so on that I might otherwise rathole on and not actually get to the project. It means I can start to see progress faster helping with my motivation.

Creating C grade skeleton #

Very similar to the above, the AI can help create a maybe C grade skeleton of the project. Its easier for me to work from this skeleton and fix things up then to start from scratch. I can run the skeleton and validate behavior as I go. I still end up rewriting maybe most of the code but I get to start with something that runs.

And for this phase of the project where I am just trying to get it to setup a skeleton I don't have to invest a lot of my focus and time. When I start a build of another project that's going to take a few minutes I can run what I have, look through the code and tell it what I want to see and how to see things setup.

When I first had it setup the project it included a bunch of popular dependencies that I really did not want. But its easy enough to go into the dependency list and remove these things and in the mean time I have a working project.

Works best when you know what you want #

This was my first time trying to write a browser extension for a real purpose. I had a number of misconceptions about how browser extensions work. This led to several instances of me trying to get Copilot to write code that was not going to work. And talking to Copilot did not help. Its too much of a 'yes' person. I ask a question in a positive way and it gives me a positive answer. I rephrase my question in a negative way and it gives me a negative answer. I had to go read MDN on exactly how things actually work for browser extension developments.

Similarly I can't tell it to do large tasks like fully describe the options page. I need to understand how I want the code structured, and go one by one through the features.

Noobs WPT in Chromium learnings

2025-03-20T00:00:00Z

I'm happy to have finished my first Chromium bug that adds a WPT. Writing a WPT I found to be more fun than writing usual C++ based tests. But I did learn a bunch from issues I ran into and otherwise that I want to record here.

Background #

The bug was deferring registerProtocolHandler duriung prerendering. You can see the spec change that led to opening this bug.

The actual bug fix was pretty simple. The existing (un)registerProtocolHandler code is already split into validation in the renderer process before calling into the browser process for the actual registration work. The bug requires doing the validation synchronously and deferring the registration work so the existing code was already setup very nicely for this change.

I found existing examples of deferring work (service worker registration, AutofillAgent) that shows using two functions to easily defer work:

Check if we're prerendering using Document::IsPrerendering
Defer work until after activation using Document::AddPostPrerenderingActivationStep

WPT learnings #

Looking in WPT I could see existing tests for prerendering and for registerProtocolHandler so my plan for testing was to add WPTs. Specifically one to ensure validation occurs synchronously, that registration is deferred and unregistration is deferred. Working on this I learned a bunch of things from docs and from hitting and resolving issues. The rest of this is just my findings in no particular order.

Wildcards #

The run_web_tests / run_wpt_tests scripts don't let you use wildcards when specifying tests to run. But you can use a parameter that lets you specify tests that include a trailing wildcard. This encourages naming related tests with common prefixes. And you can do multiple rules separated by :: (🤷)

Q:\cr\src\third_party\blink\tools\run_wpt_tests.bat -t Q:\cr\src\out\debug_x64  --isolated-script-test-filter external/wpt/html/browsers/history/the-location-interface/location-prototype*::external/wpt/html/browsers/history/the-location-interface/location-protocol*

Sync'ing between chromium and WPT #

WPT has a github repo but chromium has a great system setup to do two way sync'ing between the chromium repo and the WPT repo with a bot. When you make a CL in chromium and its getting ready to submit a bot will open a github PR to push the change to the WPT repo. There's also a system to sync WPT repo changes back to chromium.

So convenient!

Manifest update 30s #

When you call run_w(eb|pt)_tests by default it will parse the manifest JSON every time to collect a list of all tests and this takes about 30 seconds (for my machine). This is a pain, but if you haven't changed the list of tests then you skip this using the parameter --no-manifest-update.

Virtual test folders #

When referring to tests in run_wpt_tests you can refer to them by default using their path like external/wpt/speculation-rules/prerender/protocol-handler-validation.html. But some can be referred to exclusively by a virtual test folder in which case they have the virtual test folder prefix: virtual/speculation-rules-prerender-target-hint/external/wpt/speculation-rules/prerender/protocol-handler-validation.html

If a test is exclusive to a virtual test folder and you try to refer to it with the default name then run_wpt_tests says the test is skipped but won't say why. You may want to search LUCI tests for a partial name of your test to find out its full and potentially virtual name.

WPT name suffix `.https.html` #

WPT names can have meaning from tags in the name. The big one for me is the .https.html tag which indicates the test must run using https. If you don't the test will run as http. I had trouble with this because when I went to debug the test its via http://localhost which although http it is localhost and chromium throws devs a bone and lets content run as if its via https. Accordingly it took me a bit to realize the issue was the test was running via http and so registerProtocolHandler just wasn't there.

Testdriver browser extension `test_driver` #

Some WPT need to do things a browser shouldn't allow script to do normally for example skipping end user UI. For this purpose there's a Testdriver browser extension which exposes APIs to script like test_driver.set_rph_registration_mode('autoAccept'); (source). This specifically will make registerProtocolHandler not show UI to the end user and auto accept instead. I was worried about not being able to test registerProtocolHandler in WPTs before finding out about this.

Because WPT has to work in Firefox and Safari as well, adding to Testdriver may require significant effort to ensure it works in these browsers.

Debugging #

My go to for debugging a WPT in the browser is via devtools. But if you're relying on the testdriver you won't be able to. I tried a few things: * Use --enable-per-test-tracing and (as long as your test isn't timing out) you'll get pftrace files created that you can open in about:tracing / https://ui.perfetto.dev/ * Use --driver-logging / --verbose and the driver will log console.log messages to the console from any context (main test page, iframes, service workers) * Use --additional-driver-flag=--enable-logging --additional-driver-flag=--v=1 with --driver-logging to turn on additional chromium logging. But just --driver-logging will get you chromium error logging of at least --v=3 (?).

content_shell #

You can run WPT using two scripts. run_wpt_tests is the newer and preferred way to run tests using chrome or headless test driver. Tests are getting ported over to running this way.

run_web_tests is the older way that uses content_shell to run tests. You have to build content_shell specifically and it replaces some parts of the product like replacing the Browser class with the Shell class. This was a big problem for me because I didn't realize that register protocol handler's main registration code is in Browser::RegisterProtocolHandler and Browser::UnregisterProtocolHandler. Running in content_shell instead I get Shell::RegisterProtocolHandler and Shell::UnregisterProtocolHandler but the unregister method is empty. I couldn't figure out why my test wasn't working until I added logging to the unregister methods and could see what was missing.

New test stability #

When adding a new test its important to ensure there's no reliability issues. You can locally rerun a test 20 times using something like --repeat-each 20 and validate that it continues to work. When you submit the change via chromium CL it will see you are adding a new test and run it many times to ensure this as well.

I had issues with my test sometimes failing but only on Mac. I hear that there are some reliabililty issues on specific Mac setups in the test lab and so I was recommended to allow for timeout from my tests on that group of Macs.

Command line summary #

run_wpt_tests:

Q:\cr\src\third_party\blink\tools\run_wpt_tests.bat -t Q:\cr\src\out\debug_x64  virtual/speculation-rules-prerender-target-hint/external/wpt/speculation-rules/prerender/protocol-handler-unregister.https.html --verbose --num-retries 0 --no-manifest-update -p chrome --repeat-each 20

run_web_tests:

Q:\cr\src\third_party\blink\tools\run_web_tests.bat -t Q:\cr\src\out\debug_x64  virtual/speculation-rules-prerender-target-hint/external/wpt/speculation-rules/prerender/protocol-handler-unregister.https.html --verbose --details --driver-logging --num-retries 0 --no-manifest-update --enable-per-test-tracing

run_web_tests param	run_wpt_tests param	notes
`--driver-logging`	`--verbose`	extra info and logging from tests and test infra. Includes chromium LOG statements
`--verbose --details`	n/a	print extra info for every test result
`--num-retries 0`	`--num-retries 0`	Don't retry failed tests
`--repeat-each 20`	`--repeat-each 20`	Rerun every test 20 times
`--no-manifest-update`	`--no-manifest-update`	Don't rebuild test list (30s)
`--enable-per-test-tracing`	n/a	Generate pftrace files for the test run

Eleventy on GitHub Pages (Part 3)

2025-02-02T00:00:00Z

Previously, I made an Eleventy based GitHub project with GitHub automation to run Eleventy when checking in changes and hooked up a custom subdomain to the GitHub pages thinking I would start a new domain for the new blog. But I changed my mind again and now have my old URI pointing to the new blog:

I made a PowerShell script to take my old blog's atom XML and turn them into post MD files for Eleventy. It was more difficult than I thought because my atom was not valid XML (eek). Rather than write code to fix it, it was easier to just edit the XML by hand to fix.
I also customized my Eleventy URIs for blog posts to match my old blog's URIs. I added post.11tydata.js files that setup the URI using custom JS functions. I define the functions in eleventy.config.js via eleventyConfig.addFilter.
That covered most of the static blog content. For the rest I made a new subdomain to point to the old server and added a custom 404 page that can help redirect to the old server. GitHub Pages custom 404 are supported by putting a 404.html in the root path.
I followed GitHub's directions for hooking up a top level domain name which was just to make A and AAAA records pointing to the GitHub IP addresses. It took a bit to populate but then worked great.
I forgot to update the CNAME file. Doing a push reset my domain for my github pages back to the previous name which no longer points to anything and broke the site. I fixed the content in the CNAME file to address the issue.

I'm going to call this transition complete! Its so much easier to work with Eleventy to add posts and not have to worry about server upgrades breaking things. For the new web content I've finished making a sticky scroll animated header but still have thoughts for the future like a share button, and Mastodon based comments.

CSS Scroll Animated Sticky Header

2025-01-02T00:00:00Z

I've made a sticky header that scroll animates all using CSS. I'm using the new position: sticky to make my <header> stick to the top of the page as the user scrolls down. Then I'm using animation-timeline: scroll(root) to animate the header and its content shorter when scrolling down the page.

The big issue I ran into was a circular dependency: the header animates shorter based on the scroll position, and the header becoming shorter changes the document height and accordingly the scroll position. This resulted in strange jumping around while scrolling.

To resolve it I'm adjusting the margin-bottom of the header so that the overall height including the margin remains the same.

  /* default size assuming scroll animation is not supported */
  #floatyHeader {
  	top: 0px;
  	position: sticky;
  	height: 4em;
  }

  /* setup scroll animation */
	#floatyHeader {
		animation: headerChange auto linear;
		animation-timeline: scroll(root);
	}

	@keyframes headerChange {
		0% { 
			height: 300px; 
			margin-bottom: 0px; 
		}
		15% { 
			height: 4em; 
      /* set the margin-bottom to the starting size minus the end size */
			margin-bottom: calc(300px - 4em);
		}
		100% { 
			height: 4em; 
			margin-bottom: calc(300px - 4em);
		}
	}

Here's an example.

Eleventy on GitHub Pages (Part 2)

2024-05-28T00:00:00Z

Previously, I made an Eleventy based GitHub project with GitHub automation to run Eleventy when checking in changes. I stopped at that point because hooking in all my old blog's posts without changing their URIs sounded like a big pain, but now I've decided to go ahead and just link from old blog to new blog and not try to move them.

This time, I've hooked up a custom subdomain (dave.deletethis.net) to the GitHub output which was surprisingly easy:

GitHub has docs on using a custom subdomain which I followed mostly trouble free. The first step is in the GitHub project's Settings, Pages, Custom domain section to set your subdomain. However, this starts a domain check which fails because I haven't done the next step yet.
I had to edit my DNS records. Previously I had some A records for '*' subdomains which I removed because I realized they weren't in use and added a CNAME record for subdomain dave to david-risney.github.io.
After waiting a while (for the A record entries to expire) GitHub's domain check succeeded.
In step 3 GitHub will check in a file named CNAME to your gh-pages branch for you. However, I'm auto generating the contents of the gh-pages branch from my main branch. So I copy the CNAME file to the public folder. Eleventy will copy the contents of the public folder over directly.
Now that I'm using a custom domain I updated my /eleventy.config.js file's pathPrefix property to / since now the content is rooted at the root path of the domain rather than in a subfolder.
I also updated /_data/metadata.js to update the name of the blog and the URI of the blog.

After pushing all these changes I have my Eleventy GitHub blog running on a custom domain. In the future I need to update the template and see if I can start importing content from the old blog.

Eleventy on GitHub Pages

2023-01-27T00:00:00Z

I wanted to replace my existing blog solution with a simple markdown setup. I decided to give Eleventy a try and host it on GitHub Pages. Here are the steps I've taken so far:

Create a new GitHub project using Elventy Base Blog as a project template.
Used the yml from How to deploy eleventy to github pages with github actions in a new file .github/workflows/build_eleventy.yml.
I updated some version numbers, and added a permissions contents write per the peaceiris/actions-gh-pages help.
I also made sure to create the gh-pages branch.
And in the GitHub project's Settings | Pages, set Source to Deploy from a branch and Branch to gh-pages.
Then I needed to update the pathPrefix setting to /blog/ to match my project name.

With this I've got Eleventy mostly setup and running on GitHub with GitHub Pages to serve the blog and GitHub Actions to rebuild the site whenever I push. Next up, how do I find a better site layout? And after that, the much harder problem of can I move my existing blog content over to this? Maybe in the interim just merge this blog content into deletethis.net.

Restricted Capabilities and MSIX APIs

2022-09-16T00:00:00Z

SysInternals is in the Windows Store? This surprised me since last I knew you couldn't have admin-y utilities as appx packages in the Windows Store.

Taking a look at their appxmanifest, you can see how they've used restricted capabilities to get admin-like access to what they need. You can view the appxmanifest of any installed package using the Get-AppxPackage PowerShell command and the Get-AppxPackageManifest command:

> (Get-AppxPackage *SysInternals* | Get-AppxPackageManifest).Package.Capabilities.OuterXml

(After formatting the results for you dear reader)

<rescap:Capability Name="runFullTrust"/>
<rescap:Capability Name="allowElevation"/>
<rescap:Capability Name="unvirtualizedResources"/>

These three capabilities are all documented:

fullTrust is to run desktop apps from an msix package.
allowElevation is to run your desktop app elevated.
unvirtualizedResources is to skip the normal registry and disk virtualization that is usually done to help sandbox packaged apps and desktop apps from messing with one another.

But do mere mortals such as you or I get to use restricted capabilities and get your own admin utility app into the Store? The docs say there is an approval process to use restricted capabilities in the Store so... maybe with extra scrutiny?

However, the docs for allowElevation are less encouraging:

We don't recommend that you declare this capability in applications that you submit to the Microsoft Store. In most cases, the use of this capability won't be approved.

OK well, I appreciate the bluntness anyway. And elsewhere on that page it tries to steer me away

Note that you can sideload apps that declare restricted capabilities without needing to receive any approval. Approval is only required when submitting these apps to the Store.

So is it still useful to package my utility desktop app as MSIX if I'm not going to put it in the Store? Yes! There's a whole set of APIs that didn't exist last time I looked in this area. You can check for and install updates to your MSIX app. And similar docs on installing your non-Store MSIX in the first place.

(Incidentally that doc mentions how ms-appinstaller: has been disabled. I'd love to read more about what happened there but based on just the name of this URI scheme and my previous life working on URIs I'll make an educated guess.)

I started looking into all this to see if I should move WebView2Utilities over to MSIX and seems like I really should. Although until I get a proper code signing certificate seems like this isn't going to happen.

(Unless I just tell folks to do something dangerous like run a .ps1 file off the github repo that gets the msix, extracts and trusts its cert, then installs the msix...)

404 Good News, Bad News

2022-09-12T00:00:00Z

So much has to go right to get a 404. Machines all over the world talking to each other over a surprisingly large number of protocols. It's almost a disservice to call it an error.

404 Good News Bad News

When I make a browser, the default 404 page is going to be incredibly detailed:

Sorry we couldn't locate your resource, but a number of things went well to get you this far: DNS, TCP/IP connection, the Bombe helped crack Nazi encryption and win WW2, metallurgy invented before at least 5500BC, ...

Watch JavaScript via Proxy

2022-02-24T00:00:00Z

JavaScript has a type Proxy that lets you intercept all interactions with an object - all property reads, writes, method calls, and so on. One fun thing you can do with this is watch how JavaScript itself interacts with your objects.

If we put a Proxy in another Proxy we can see how JavaScript uses our objects:

function trace(obj, name) {
    return new Proxy(obj, new Proxy({}, { 
        get: (t, property, r) => 
            logCall.bind(null, name + ' ' + property, Reflect[property], Reflect)
    }));
}

function logCall(description, fn, fnThis, ...fnArgs) {
    try {
        const r = fn.call(fnThis, ...fnArgs);
        console.log(description + '\n\t', ...fnArgs, "\n  -->", r);
        return r;
    } catch (e) {
        console.log(description + '\n\t', ...fnArgs, "\n  -fail->", e);
        throw e;
    }
}

How this works #

trace returns a Proxy around your object that uses Reflect to perform the normal Proxy operations but then console.log the call.

The constructor for Proxy takes two parameters: first the object you're wrapping, and second an object with functions for all the different kind of object interactions that you want to intercept. Because we want to just log the interaction and not do anything specific we can use a Proxy around Reflect to define this second parameter.

Our inner Proxy implements the handler of the outer Proxy. The handler is supposed to have methods called get, set, apply and others for each operation to intercept. Instead we define our inner Proxys get to return a function that will perform the operation and log it using logCall. The function we pass in to logCall to run is one of the functions off of Reflect. Reflect defines all the Proxy handler functions get, set, apply and so on but where each just performs those operations. Reflect is like a convenient default for all Proxy handler functions. We want to perform the default operations but additionally log the result so using Reflect is exactly what we want.

What you can do with it #

With that you can call trace with an object and get out a Proxy that acts just like your object but also console.logs every get, set, has, getOwnPropertyDescriptor, and so on that JavaScript asks of your object.

You can watch how Array.sort sorts an array.

Or how Array.toString calls Array.join.

Or how Array.concat checks for Symbol.isConcatSpreadable.

Check out the corresponding ECMAScript standard and you can see how the logged Proxy calls match the standard defined steps for things like Array.sort.

Because I pass objects to console.log DevTools will show you an interactive object in the console. If you try it out the log is easier to understand than as text in the gist:

array get 
        (3) [300, 200, 100] sort Proxy {0: 300, 1: 200, 2: 100} ) 
    --> ƒ sort() { [native code] }

The above is a Proxy handler's get call. The first param is the array (3) [300, 200, 100] on which we will 'get'. The second param is the string 'sort' that is the property to get. The third is our proxy.

I haven't tried wrapping the return values in trace Proxy although maybe that would give an even better picture of what's happening.

Various Array functions can operate on a this that is a not an Array. If you're trying to make a non-Array object work with Array functions, then using a Proxy to watch what JavaScript is doing can help debug.

Watch a `for` loop #

But Proxy watching works for other parts of JavaScript and not just standard functions. For example watch a for loop enumerate over your array:

let arr = trace([300, 200, 100], 'array'); 
for (let name in arr) { }

And the log:

array ownKeys
    (3) [300, 200, 100] 
  -->  (4) ['0', '1', '2', 'length']
array getPrototypeOf
    (3) [300, 200, 100] 
  -->  [constructor: ƒ, at: ƒ, concat: ƒ, copyWithin: ƒ, fill: ƒ, …]
array getOwnPropertyDescriptor
    (3) [300, 200, 100] 0 
  -->  {value: 300, writable: true, enumerable: true, configurable: true}
array getOwnPropertyDescriptor
    (3) [300, 200, 100] 1 
  -->  {value: 200, writable: true, enumerable: true, configurable: true}
array getOwnPropertyDescriptor
    (3) [300, 200, 100] 2 
  -->  {value: 100, writable: true, enumerable: true, configurable: true}
array getOwnPropertyDescriptor
    (3) [300, 200, 100] length 
  -->  {value: 3, writable: true, enumerable: false, configurable: false}

The for loop asks your object for the names of its properties and then the value of each.

Watch `instanceof` #

Or instanceof with your proxy on the left hand side

arr instanceof String

And watch JavaScript get the prototype of array

array getPrototypeOf
    (3) [300, 200, 100] 
  -->  [constructor: ƒ, at: ƒ, concat: ƒ, copyWithin: ƒ, fill: ƒ, …]

Or instanceof with your proxy on the right hand side

String instanceof arr

And watch JavaScript ask your array for its Symbol.hasInstance property.

array get
    (3) [300, 200, 100] Symbol(Symbol.hasInstance) Proxy(Array) {0: 300, 1: 200, 2: 100} 
  -->  undefined

(Here's a related gist.)

JavaScript Arrays are neat

2021-11-24T00:00:00Z

JavaScript arrays are unbelievable. If I told you the length property of an array is writable you might chuckle and say 'oh JavaScript'. But no, it goes all the way.

const a = [1,2,3,4,5];
a.length = 3;
console.log(a.length); // 3
console.log(a.join(',')); // 1,2,3

This seems like a curve where folks very new to programming or very experienced with JavaScript know what this does and folks in the middle say that can't possibly work:

--a.length;

As you'd expect given what decreasing length does, increasing length works similarly:

const a = [1,2,3,4,5];
a.length = 7;
console.log(a.length); // 7
console.log(a.join(',')); // 1,2,3,4,5,,

The length property is not configurable so you can't delete it. There's something I like about telling JavaScript to delete length and its just like, 'nah'.

let a = [1,2,3];
delete a.length; // false

More fun: The Array functions don't care if they act on an Array. They only require an object with numbered properties and a length property:

let o = {'0':'b','1':'a','length':2};
Array.prototype.sort.call(o); // {0: 'a', 1: 'b', length: 2}
Array.prototype.slice.call(o, 1); // ['b']

Now that I have an object with a configurable length property I can delete the length. After that the Array functions don't like my object anymore. But they don't complain. They still produce something:

delete o.length;
Array.prototype.slice.call(o, 1); // []
Array.prototype.join.call(o, ','); // ''

Right-To-Left Override Twitter Name

2020-10-21T00:00:00Z

Its rare to find devs anticipating Unicode control characters showing up in user input. And the most fun when unanticipated is the Right-To-Left Override character U+202E. Unicode characters have an implicit direction so that for example by default Hebrew characters are rendered from right to left, and English characters are rendered left to right. The override characters force an explicit direction for all the text that follows.

I chose my Twitter display name to include the HTML encoding of the Right-To-Left Override character #x202E; as a sort of joke or shout out to my favorite Unicode control character. I did not anticipate that some Twitter clients in some of their UI would fail to encode it correctly. There's no way I can remove that from my display name now.

Try it on Amazon.

How about pages that want to tell you about the U+202E.

Scrollbars in EdgeHtml WebView and Edge browser

2019-08-22T00:00:00Z

The scrollbars in UWP WebView and in Edge have different default behavior leading to many emails to my team. (Everything I talk about here is for the EdgeHtml based WebView and Edge browser and does not apply to the Chromium based Edge browser and WebView2).

There is a Edge only -ms-overflow-style CSS property that controls scroll behavior. We have a different default for this in the WebView as compared to the Edge browser. If you want the appearance of the scrollbar in the WebView to match the browser then you must explicitly set that CSS property. The Edge browser default is scrollbar which gives us a Windows desktop styled non-auto-hiding scrollbar. The WebView default is -ms-autohiding-scrollbar which gives a sort of compromise between desktop and UWP app scrollbar behavior. In this configuration it is auto-hiding. When used with the mouse you'll get Windows desktop styled scrollbars and when used with touch you'll get the UWP styled scrollbars.

Since WebViews are intended to be used in apps this style is the default in order to better match the app's scrollbars. However this difference between the browser and WebView has led to confusion.

Here’s an -ms-overflow-style JSFiddle showing the difference between the two styles. Try it in the Edge browser and in WebView. An easy way to try it in the Edge WebView is using the JavaScript Browser.

Edge browser and JavaScript UWP app security model comparison

2018-11-29T00:00:00Z

There are two main differences in terms of security between a JavaScript UWP app and the Edge browser:

Process Model

A JavaScript UWP app has one process (technically not true with background tasks and other edge cases but ignoring that for the moment) that runs in the corresponding appcontainer defined by the app's appx manifest. This one process is where edgehtml is loaded and is rendering HTML, talking to the network, and executing script. Specifically, the UWP main UI thread is the one where your script is running and calling into WinRT.

In the Edge browser there is a browser process running in the same appcontainer defined by its appx manifest, but there are also tab processes. These tab processes are running in restricted app containers that have fewer appx capabilities. The browser process has XAML loaded and coordinates between tabs and handles some (non-WinRT) brokering from the tab processes. The tab processes load edgehtml and that is where they render HTML, talk to the network and execute script.

There is no way to configure the JavaScript UWP app's process model but using WebViews you can approximate it. You can create out of process WebViews and to some extent configure their capabilities, although not to the same extent as the browser. The WebView processes in this case are similar to the browser's tab processes. See the MSWebViewProcess object for configuring out of process WebView creation. I also implemented out of proc WebView tabs in my JSBrowser fork.

ApplicationContentUriRules

The ApplicationContentUriRules (ACUR) section of the appx manifest lets an application define what URIs are considered app code. See a previous post for the list of ACUR effects.

Notably app code is able to access WinRT APIs. Because of this, DOM security restrictions are loosended to match what is possible with WinRT.

Privileged DOM APIs like geolocation, camera, mic etc require a user prompt in the browser before use. App code does not show the same browser prompt. There still may be an OS prompt – the same prompt that applies to any UWP app, but that’s usually per app not per origin.

App code also gets to use XMLHttpRequest or fetch to access cross origin content. Because UWP apps have separate state, cross origin here might not mean much to an attacker unless your app also has the user login to Facebook or some other interesting cross origin target.

GoBack/GoForward in Win10 UWP WebView

2018-10-23T00:00:00Z

The GoBack and GoForward methods on the UWP WebView (x-ms-webview in HTML, Windows.UI.Xaml.Controls.WebView in XAML, and Windows.Web.UI.Interop.WebViewControl in Win32) act the same as the Back and Forward buttons in the Edge browser. They don't necessarily change the top level document of the WebView. If inside the webview an iframe navigates then that navigation will be recorded in the forward/back history and the GoBack / GoForward call may result in navigating that iframe. This makes sense as an end user using the Edge browser since if I click a link to navigate one place and then hit Back I expect to sort of undo that most recent navigation regardless of if that navigation happened in an iframe or the top level document.

If that doesn't make sense for your application and you want to navigate forward or back ignoring iframe navigates, unfortunately there's no perfect workaround.

One workaround could be to try calling GoBack and then checking if a FrameNavigationStarting event fires or a NavigationStarting event fires. If a frame navigates then try calling GoBack again. There could be async races in this case since other navigates could come in and send you the wrong signal and interrupt your multi step GoBack operation.

You could also try keeping track of all top level document navigations and manually navigate back to the URIs you care about. However, GoBack and GoForward also restore some amount of user state (form fills etc) in addition to navigating. Manually calling navigate will not give this same behavior.

Changing the User Agent string in UWP WebView

2018-10-23T00:00:00Z

There's no perfect way to change the user agent string for the UWP WebView (x-ms-webview in HTML, Windows.UI.Xaml.Controls.WebView in XAML, and Windows.Web.UI.Interop.WebViewControl in Win32) but there are two imperfect methods folks end up using.

The first is to call UrlMkSetSessionOption. This is an old public API that allows you to configure various arcane options including one that is the default user agent string for requests running through urlmon. This API is allowed by the Microsoft Store for UWP apps. The change it applies is process wide which has two potential drawbacks. If you want to be able to have different UA strings set for different requests from a WebView that's not really possible with this solution. The other drawback is if you're using out of process WebView, you need to ensure you're calling into UrlMkSetSessionOption in the WebView's process. You'll need to write third party WinRT that calls UrlMkSetSessionOption, create the out of proc WebView, navigate it to some trusted local page, use AddWebAllowedObject or provide that URI WinRT access, and call into your third party WinRT. You'll need to do that for any new WebView process you create.

The second less generally applicable solution is to use NavigateWithHttpRequestMessage and set the User-Agent HTTP header. In this case you get to control the scope of the user agent string changes but has the limitations that not all sub resource downloads will use this user agent string and for navigations you don't initiate you have to manually intercept and re-request being careful to transfer over all POST body state and HTTP headers correctly. That last part is not actually possible for iframes.

Windows.Web.UI.Interop.WebViewControl localhost access

2018-07-25T00:00:00Z

If you're developing with the new Windows.Web.UI.Interop.WebViewControl you may have noticed you cannot navigate to localhost HTTP servers. This is because the WebViewControl's WebView process is a UWP process. All UWP processes by default cannot use the loopback adapter as a security precaution. For development purposes you can allow localhost access using the checknetisolation command line tool on the WebViewControl's package just as you can for any other UWP app. The command should be the following:

checknetisolation loopbackexempt -a -n=Microsoft.Win32WebViewHost_cw5n1h2txyewy

As a warning checknetisolation is not good on errors. If you attempt to add a package but get its package family name wrong, checknetisolation just says OK:

C:\Users\davris>checknetisolation LoopbackExempt -a -n=Microsoft.BingWeather_4.21.2492.0_x86__8wekyb3d8bbwe
OK.

And if you then list the result of the add with the bad name you'll see the following:

[1] -----------------------------------------------------------------
    Name: AppContainer NOT FOUND
    SID:  S-1-15-...

There's also a UI tool for modifying loopback exemption for packages available on GitHub and also one available with Fiddler.

As an additional note, I mentioned above you can try this for development. Do not do this in shipping products as this turns off the security protection for any consumer of the WebViewControl.

Win10 PWA Terminology

2018-05-31T00:00:00Z

Folks familiar with JavaScript UWP apps in Win10 have often been confused by what PWAs in Win10 actually are. TLDR: PWAs in Win10 are simply JavaScript UWP apps. The main difference between these JS UWP Apps and our non-PWA JS UWP apps are our target end developer audience, and how we get Win10 PWAs into the Microsoft Store. See this Win10 blog post on PWAs on Win10 for related info.

Web App

On the web a subset of web sites are web apps. These are web sites that have app like behavior - that is a user might call it an app like Outlook, Maps or Gmail. And they may also have a W3C app manifest.

A subset of web apps are progressive web apps. Progressive web apps are web apps that have a W3C app manifest and a service worker. Various OSes are beginning to support PWAs as first class apps on their platform. This is true for Win10 as well in which PWAs are run as a WWA.

Windows Web App

In Win10 a WWA (Windows Web App) is an unofficial term for a JavaScript UWP app. These are UWP apps so they have an AppxManifest.xml, they are packaged in an Appx package, they run in an App Container, they use WinRT APIs, and are installed via the Microsoft Store. Specific to WWAs though, is that the AppxManifest.xml specifies a StartPage attribute identifying some HTML content to be used as the app. When the app is activated the OS will create a WWAHost.exe process that hosts the HTML content using the EdgeHtml rendering engine.

Packaged vs Hosted Web App

Within that we have a notion of a packaged web app and an HWA (hosted web app). There's no real technical distinction for the end developer between these two. The only real difference is whether the StartPage identifies remote HTML content on the web (HWA), or packaged HTML content from the app's appx package (packaged web app). An end developer may create an app that is a mix of these as well, with HTML content in the package and HTML content from the web. These terms are more like ends on a continuum and identifying two different developer scenarios since the underlying technical aspect is pretty much identical.

Win10 PWA

Win10 PWAs are simply HWAs that specify a StartPage of a URI for a PWA on the web. These are still JavaScript UWP apps with all the same behavior and abilities as other UWP apps. We have two ways of getting PWAs into the Microsoft Store as Win10 PWAs. The first is PWA Builder which is a tool that helps PWA end developers create and submit to the Microsoft Store a Win10 PWA appx package. The second is a crawler that runs over the web looking for PWAs which we convert and submit to the Store using an automated PWA Builder-like tool to create a Win10 PWA from PWAs on the web (see Welcoming PWAs to Win10 for more info). In both cases the conversion involves examining the PWAs W3C app manifest and producing a corresponding AppxManifest.xml. Not all features supported by AppxManifest.xml are also available in the W3c app manifest. But the result of PWA Builder can be a working starting point for end developers who can then update the AppxManifest.xml as they like to support features like share targets or others not available in W3C app manifests.

Tiny browser features: JSBrowser crash resistance

2018-05-13T00:00:00Z

JSBrowser is a basic browser built as a Win10 JavaScript UWP app around the WebView HTML element. Its fun and relatively simple to implement tiny browser features in JavaScript and in this post I'm implementing crash resistance.

The normal DOM mechanisms for creating an HTML WebView create an in-process WebView, in which the WebView runs on a unique UI thread. But we can use the MSWebView constructor instead to create an out-of-process WebView in which the WebView runs in its own distinct WebView process. Unlike an in-process WebView, Web content running in an out-of-process WebView can only crash the WebView process and not the app process.

        this.replaceWebView = () => {
            let webview = document.querySelector("#WebView");
            // Cannot access webview.src - anything that would need to communicate with the webview process may fail
            let oldSrc = browser.currentUrl;
            const webviewParent = webview.parentElement;
            webviewParent.removeChild(webview);
            webview = new MSWebView();
            Object.assign(this, {
                "webview": webview
            });
            webview.setAttribute("id", "WebView");

            // During startup our currentUrl field is blank. If the WebView has crashed 
            // and we were on a URI then we may obtain it from this property.
            if (browser.currentUrl && browser.currentUrl != "") {
                this.trigger("newWebview");
                this.navigateTo(browser.currentUrl);
            }
            webviewParent.appendChild(webview);

I run replaceWebView during startup to replace the in-process WebView created via HTML markup with an out-of-process WebView. I could be doing more to dynamically copy styles, attributes, etc but I know what I need to set on the WebView and just do that.

When a WebView process crashes the corresponding WebView object is no longer useful and a new WebView element must be created. In fact if the old WebView object is used it may throw and will no longer have valid state. Accordingly when the WebView crashes I run replaceWebView again. Additionally, I need to store the last URI we've navigated to (browser.currentUrl in the above) since the crashed WebView object won't know what URI it is on after it crashes.

            webview.addEventListener("MSWebViewProcessExited", () => { 
                if (browser.currentUrl === browser.lastCrashUrl) {                   ++browser.lastCrashUrlCrashCount; 
                } 
                else { 
                    browser.lastCrashUrl = browser.currentUrl; 
                    browser.lastCrashUrlCrashCount = 1; 
                } 
                // If we crash again and again on the same URI, maybe stop trying to load that URI. 
                if (browser.lastCrashUrlCrashCount >= 3) { 
                    browser.lastCrashUrl = ""; 
                    browser.lastCrashUrlCrashCount = 0; 
                    browser.currentUrl = browser.startPage; 
                } 
                this.replaceWebView(); 
            });

I also keep track of the last URI that we recovered and how many times we've recovered that same URI. If the same URI crashes more than 3 times in a row then I assume that it will keep happening and I navigate to the start URI instead.

Tiny browser features: JSBrowser zoom

2018-05-10T00:00:00Z

My plan to implement zoom is to add a zoom slider to the settings div that controls the scale of the WebView element via CSS transform. My resulting zoom change is in git and you can try the whole thing out in my JSBrowser fork.

Slider

I can implement the zoom settings slider as a range type input HTML element. This conveniently provides me a min, max, and step property and suits exactly my purposes. I chose some values that I thought would be reasonable so the browser can scale between half to 3x by increments of one quarter. This is a tiny browser feature after all so there's no custom zoom entry.

<a><label for="webviewZoom">Zoom</label><input type="range" min="50" max="300" step="25" value="100" id="webviewZoom" /></a>

To let the user know this slider is for controlling zoom, I make a label HTML element that says Zoom. The label HTML element has a for attribute which takes the id of another HTML element. This lets the browser know what the label is labelling and lets the browser do things like when the label is clicked to put focus on the slider.

Scale

There are no explicit scale APIs for WebView so to change the size of the content in the WebView we use CSS.

        this.applyWebviewZoom = state => {
            const minValue = this.webviewZoom.getAttribute("min");
            const maxValue = this.webviewZoom.getAttribute("max");
            const scaleValue = Math.max(Math.min(parseInt(this.webviewZoom.value, 10), maxValue), minValue) / 100;

            // Use setAttribute so they all change together to avoid weird visual glitches
            this.webview.setAttribute("style", [
                ["width", (100 / scaleValue) + "%"],
                ["height", "calc(" + (-40 / scaleValue) + "px + " + (100 / scaleValue) + "%)"],
                ["transform", "scale(" + scaleValue + ")"]
            ].map(pair => pair[0] + ": " + pair[1]).join("; "));
        };

Because the user changes the scale at runtime I accordingly replace the static CSS for the WebView element with the script above to programmatically modify the style of the WebView. I change the style with one setAttribute call to do my best to avoid the browser performing unnecessary work or displaying the WebView in an intermediate and incomplete state. Applying the scale to the element is as simple as adding 'transform: scale(X)' but then there are two interesting problems.

The first is that the size of the WebView is also scaled not just the content within it. To keep the WebView the same effective size so that it still fits properly into our browser UI, we must compensate for the scale in the WebView width and height. Accordingly, you can see that we scale up by scaleValue and then in width and height we divide by the scaleValue.

transform-origin: 0% 0%;

The other issue is that by default the scale transform's origin is the center of the WebView element. This means when scaled up all sides of the WebView would expand out. But when modifying the width and height those apply relative to the upper left of the element so our inverse scale application to the width and height above aren't quite enough. We also have to change the origin of the scale transform to match the origin of the changes to the width and height.

Multiple Windows in Win10 JavaScript UWP apps

2018-03-10T00:00:00Z

Win10 Changes

In Win8.1 JavaScript UWP apps we supported multiple windows using MSApp DOM APIs. In Win10 we use window.open and window and a new MSApp API getViewId and the previous MSApp APIs are gone:

	Win10	Win8.1
Create new window	window.open	MSApp.createNewView
New window object	window	MSAppView
viewId	MSApp.getViewId(window)	MSAppView.viewId

WinRT viewId

We use window.open and window for creating new windows, but then to interact with WinRT APIs we add the MSApp.getViewId API. It takes a window object as a parameter and returns a viewId number that can be used with the various Windows.UI.ViewManagement.ApplicationViewSwitcher APIs.

Delaying Visibility

Views in WinRT normally start hidden and the end developer uses something like TryShowAsStandaloneAsync to display the view once it is fully prepared. In the web world, window.open shows a window immediately and the end user can watch as content is loaded and rendered. To have your new windows act like views in WinRT and not display immediately we have added a window.open option. For example
let newWindow = window.open("https://example.com", null, "msHideView=yes");

Primary Window Differences

The primary window that is initially opened by the OS acts differently than the secondary windows that it opens:

	Primary	Secondary
window.open	Allowed	Disallowed
window.close	Close app	Close window
Navigation restrictions	ACUR only	No restrictions

The restriction on secondary windows such that they cannot open secondary windows could change in the future depending on feedback.

Same Origin Communication Restrictions

Lastly, there is a very difficult technical issue preventing us from properly supporting synchronous, same-origin, cross-window, script calls. That is, when you open a window that's same origin, script in one window is allowed to directly call functions in the other window and some of these calls will fail. postMessage calls work just fine and is the recommended way to do things if that's possible for you. Otherwise we continue to work on improving this.

MSApp.getHtmlPrintDocumentSourceAsync - JavaScript UWP app printing

2017-10-11T00:00:00Z

The documentation for printing in JavaScript UWP apps is out of date as it all references MSApp.getHtmlPrintDocumentSource but that method has been replaced by MSApp.getHtmlPrintDocumentSourceAsync since WinPhone 8.1.

Background

Previous to WinPhone 8.1 the WebView's HTML content ran on the UI thread of the app. This is troublesome for rendering arbitrary web content since in the extreme case the JavaScript of some arbitrary web page might just sit in a loop and never return control to your app's UI. With WinPhone 8.1 we added off thread WebView in which the WebView runs HTML content on a separate UI thread.

Off thread WebView required changing our MSApp.getHtmlPrintDocumentSource API which could no longer synchronously produce an HtmlPrintDocumentSource. With WebViews running on their own threads it may take some time for them to generate their print content for the HtmlPrintDocumentSource and we don't want to hang the app's UI thread in the interim. So the MSApp.getHtmlPrintDocumentSource API was replaced with MSApp.getHtmlPrintDocumentSourceAsync which returns a promise the resolved value of which is the eventual HtmlPrintDocumentSource.

Sample

However, the usage of the API is otherwise unchanged. So in sample code you see referencing MSApp.getHtmlPrintDocumentSource the sample code is still reasonable but you need to call MSApp.getHtmlPrintDocumentSourceAsync instead and wait for the promise to complete. For example the PrintManager docs has an example implementing a PrintTaskRequested event handler in a JavaScript UWP app.

    function onPrintTaskRequested(printEvent) {    
        var printTask = printEvent.request.createPrintTask("Print Sample", function (args) {
            args.setSource(MSApp.getHtmlPrintDocumentSource(document));
    });

Instead we need to obtain a deferral in the event handler so we can asynchronously wait for getHtmlPrintDocumentSourceAsync to complete:

    function onPrintTaskRequested(printEvent) {    
        var printTask = printEvent.request.createPrintTask("Print Sample", function (args) {
            const deferral = args.getDeferral();
            MSApp.getHtmlPrintDocumentSourceAsync(document).then(htmlPrintDocumentSource => {
                args.setSource(htmlPrintDocumentSource);
                deferral.complete();
            }, error => {
                console.error("Error: " + error.message + " " + error.stack);
                deferral.complete();
            });
        });

Win10 UWP WebView AddWebAllowedObject details

2017-09-04T00:00:00Z

The x-ms-webview HTML element has the void addWebAllowedObject(string name, any value) method and the webview XAML element has the void AddWebAllowedObject(String name, Object value) method. The object parameter is projected into the webview’s top-level HTML document’s script engine as a new property on the global object with property name set to the name parameter. It is not injected into the current document but rather it is projected during initialization of the next top-level HTML document to which the webview navigates.

Lifetime

If AddWebAllowedObject is called during a NavigationStarting event handler the object will be injected into the document resulting from the navigation corresponding to that event.

If AddWebAllowedObject is called outside of the NavigationStarting event handler it will apply to the navigation corresponding to the next explicit navigate method called on the webview or the navigation corresponding to the next NavigationStarting event handler that fires, whichever comes first.

To avoid this potential race, you should use AddWebAllowedObject in one of two ways: 1. During a NavigationStarting event handler, 2. Before calling a Navigate method and without returning to the main loop.

If called both before calling a navigate method and in the NavigationStarting event handler then the result is the aggregate of all those calls.

If called multiple times for the same document with the same name the last call wins and the previous are silently ignored.

If AddWebAllowedObject is called for a navigation and that navigation fails or redirects to a different URI, the AddWebAllowedObject call is silently ignored.

After successfully adding an object to a document, the object will no longer be projected once a navigation to a new document occurs.

WinRT access

If AddWebAllowedObject is called for a document with All WinRT access then projection will succeed and the object will be added.

If AddWebAllowedObject is called for a document which has a URI which has no declared WinRT access via ApplicationContentUriRules then Allow for web only WinRT access is given to that document.

If the document has Allow for web only WinRT access then projection will succeed only if the object’s runtimeclass has the Windows.Foundation.Metadata.AllowForWeb metadata attribute.

Object requirements

The object must implement the IAgileObject interface. Because the XAML and HTML webview elements run on ASTA view threads and the webview’s content’s JavaScript thread runs on another ASTA thread a developer should not create their non-agile runtimeclass on the view thread. To encourage end developers to do this correctly we require the object implements IAgileObject.

Property name

The name parameter must be a valid JavaScript property name, otherwise the call will fail silently. If the name is already a property name on the global object, that property is overwritten if the property is configurable. Non-configurable properties on the global object are not overwritten and the AddWebAllowedObject call fails silently. On success, the projected property is writable, configurable, and enumerable.

Errors

Some errors as described above fail silently. Other issues, such as lack of IAgileObject or lack of the AllowForWeb attribute result in an error in the JavaScript developer console.

Application Content URI Rule effects

2017-06-30T00:00:00Z

Previously I described Application Content URI Rules (ACUR) parsing and ACUR ordering. This post describes what you get from putting a URI in ACUR.

URIs in the ACUR gain the following which is otherwise unavailable:

Geoloc API usage
Audio and video capture API usage
Pointer lock API usage
Web notifications API usage
IndexedDB API usage
Clipboard API usage
window.external.notify access from within webview
window.close the primary window
Top level navigation in the primary window
Cross origin XHR and fetch to ms-appx(-web) scheme URIs
Cross origin dirtied canvas read access if dirtied by ms-appx(-web) scheme URIs
Cross origin text track for video element for tracks from ms-appx(-web) scheme URIs

URIs in the ACUR that also have full WinRT access additionally gain the following:

Cross origin XHR and fetch
Cross origin dirtied canvas read access
Cross origin text track for video element
Local audio and video WinRT plugins work with media elements

JavaScript Microsoft Store app StartPage

2017-06-22T00:00:00Z

JavaScript Microsoft Store apps have some details related to activation that are specific to JavaScript Store apps and that are poorly documented which I’ll describe here.

StartPage syntax

The StartPage attributes in the AppxManifest.xml (Package/Applications/Application/@StartPage, Package/Applications/Extensions/Extension/@StartPage) define the HTML page entry point for that kind of activation. That is, Application/@StartPage defines the entry point for tile activation, Extension[@Category="windows.protocol"]/@StartPage defines the entry point for URI handling activation, etc. There are two kinds of supported values in StartPage attributes: relative Windows file paths and absolute URIs. If the attribute doesn’t parse as an absolute URI then it is instead interpreted as relative Windows file path.

This implies a few things that I’ll declare explicitly here. Windows file paths, unlike URIs, don’t have a query or fragment, so if you are using a relative Windows file path for your StartPage attribute you cannot include anything like ‘?param=value’ at the end. Absolute URIs use percent-encoding for reserved characters like ‘%’ and ‘#’. If you have a ‘#’ in your HTML filename then you need to percent-encode that ‘#’ for a URI and not for a relative Windows file path.

If you specify a relative Windows file path, it is turned into an ms-appx URI by changing all backslashes to forward slashes, percent-encoding reserved characters, and combining the result with a base URI of ms-appx:///. Accordingly the relative Windows file paths are relative to the root of your package. If you are using a relative Windows file path as your StartPage and need to switch to using a URI so you can include a query or fragment, you can follow the same steps above.

StartPage validity

The validity of the StartPage is not determined before activation. If the StartPage is a relative Windows file path for a file that doesn’t exist, or an absolute URI that is not in the Application Content URI Rules, or something that doesn’t parse as a Windows file path or URI, or otherwise an absolute URI that fails to resolve (404, bad hostname, etc etc) then the JavaScript app will navigate to the app’s navigation error page (perhaps more on that in a future blog post). Just to call it out explicitly because I have personally accidentally done this: StartPage URIs are not automatically included in the Application Content URI Rules and if you forget to include your StartPage in your ACUR you will always fail to navigate to that StartPage.

StartPage navigation

When your app is activated for a particular activation kind, the StartPage value from the entry in your app’s manifest that corresponds to that activation kind is used as the navigation target. If the app is not already running, the app is activated, navigated to that StartPage value and then the Windows.UI.WebUI.WebUIApplication activated event is fired (more details on the order of various events in a moment). If, however, your app is already running and an activation occurs, we navigate or don’t navigate to the corresponding StartPage depending on the current page of the app. Take the app’s current top level document’s URI and if after removing the fragment it already matches the StartPage value then we won’t navigate and will jump straight to firing the WebUIApplication activated event.

Since navigating the top-level document means destroying the current JavaScript engine instance and losing all your state, this behavior might be a problem for you. If so, you can use the MSApp.pageHandlesAllApplicationActivations(true) API to always skip navigating to the StartPage and instead always jump straight to firing the WebUIApplication activated event. This does require of course that all of your pages all handle all activation kinds about which any part of your app cares.

Application Content URI Rules rule ordering

2017-06-01T00:00:00Z

Application Content URI Rules (ACUR from now on) defines the bounds on the web that make up a Microsoft Store application. The previous blog post discussed the syntax of the Rule's Match attribute and this time I'll write about the interactions between the Rules elements.

Order

A single ApplicationContentUriRules element may have up to 100 Rule child elements. When determining if a navigation URI matches any of the ACUR the last Rule in the list with a matching match wildcard URI is used. If that Rule is an include rule then the navigation URI is determined to be an application content URI and if that Rule is an exclude rule then the navigation rule is not an application content URI. For example:

Rule Type='include' Match='https://example.com/'/
Rule Type='exclude' Match='https://example.com/'/

Given the above two rules in that order, the navigation URI https://example.com/ is not an application content URI because the last matching rule is the exclude rule. Reverse the order of the rules and get the opposite result.

WindowsRuntimeAccess

In addition to determining if a navigation URI is application content or not, a Rule may also confer varying levels of WinRT access via the optional WindowsRuntimeAccess attribute which may be set to 'none', 'allowForWeb', or 'all'. If a navigation URI matches multiple different include rules only the last rule is applied even as it applies to the WindowsRuntimeAccess attribute. For example:

Rule Type='include' Match='https://example.com/' WindowsRuntimeAccess='none'/
Rule Type='include' Match='https://example.com/' WindowsRuntimeAccess='all'/

Given the above two rules in that order, the navigation URI https://example.com/ will have access to all WinRT APIs because the last matching rule wins. Reverse the rule order and the navigation URI https://example.com/ will have no access to WinRT. There is no summation or combining of multiple matching rules - only the last matching rule wins.

Application Content URI Rules wildcard syntax

2017-05-31T00:00:00Z

Application Content URI Rules (ACUR from now on) defines the bounds of the web that make up the Microsoft Store application. Package content via the ms-appx URI scheme is automatically considered part of the app. But if you have content on the web via http or https you can use ACUR to declare to Windows that those URIs are also part of your application. When your app navigates to URIs on the web those URIs will be matched against the ACUR to determine if they are part of your app or not. The documentation for how matching is done on the wildcard URIs in the ACUR Rule elements is not very helpful on MSDN so here are some notes.

Rules

You can have up to 100 Rule XML elements per ApplicationContentUriRules element. Each has a Match attribute that can be up to 2084 characters long. The content of the Match attribute is parsed with CreateUri and when matching against URIs on the web additional wildcard processing is performed. I’ll call the URI from the ACUR Rule the rule URI and the URI we compare it to found during app navigation the navigation URI.

The rule URI is matched to a navigation URI by URI component: scheme, username, password, host, port, path, query, and fragment. If a component does not exist on the rule URI then it matches any value of that component in the navigation URI. For example, a rule URI with no fragment will match a navigation URI with no fragment, with an empty string fragment, or a fragment with any value in it.

Asterisk

Each component except the port may have up to 8 asterisks. Two asterisks in a row counts as an escape and will match 1 literal asterisk. For scheme, username, password, query and fragment the asterisk matches whatever it can within the component.

Host

For the host, if the host consists of exactly one single asterisk then it matches anything. Otherwise an asterisk in a host only matches within its domain name label. For example, http://*.example.com will match http://a.example.com/ but not http://b.a.example.com/ or http://example.com/. And http://*/ will match http://example.com, http://a.example.com/, and http://b.a.example.com/. However the Store places restrictions on submitting apps that use the http://* rule or rules with an asterisk in the second effective domain name label. For example, http://*.com is also restricted for Store submission.

Path

For the path, an asterisk matches within the path segment. For example, http://example.com/a/*/c will match http://example.com/a/b/c and http://example.com/a//c but not http://example.com/a/b/b/c or http://example.com/a/c

Additionally for the path, if the path ends with a slash then it matches any path that starts with that same path. For example, http://example.com/a/ will match http://example.com/a/b and http://example.com/a/b/c/d/e/, but not http://example.com/b/.

If the path doesn’t end with a slash then there is no suffix matching performed. For example, http://example.com/a will match only http://example.com/a and no URIs with a different path.

As a part of parsing the rule URI and the navigation URI, CreateUri will perform URI normalization and so the hostname and scheme will be made lower case (casing matters in all other parts of the URI and case sensitive comparisons will be performed), IDN normalization will be performed, ‘.’ and ‘..’ path segments will be resolved and other normalizations as described in the CreateUri documentation.

Let's Encrypt NearlyFreeSpeech.net Update

2016-11-05T00:00:00Z

Since I had last posted about using Let's Encrypt with NearlyFreeSpeech, NFS has changed their process for setting TLS info. Instead of putting the various files in /home/protected/ssl and submitting an assistance request, now there is a command to submit the certificate info and a webpage for submitting the certificate info.

The webpage is https://members.nearlyfreespeech.net/{username}/sites/{sitename}/add_tls and has a textbox for you to paste in all the cert info in PEM form into the textbox. The domain key, the domain certificate, and the Let's Encrypt intermediate cert must be pasted into the textbox and submitted.

Alternatively, that same info may be provided as standard input to nfsn -i set-tls

To renew my certificate with the updated NFS process I followed the commands from Andrei Damian-Fekete's script which depends on acme_tiny.py:

python acme_tiny.py --account-key account.key --csr domain.csr --acme-dir /home/public/.well-known/acme-challenge/ > signed.crt
wget -O - https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem > intermediate.pem
cat domain.key signed.crt intermediate.pem > chained.pem
nfsn -i set-tls < chained.pem

Because my certificate had already expired I needed to comment out the section in acme_tiny.py that validates the challenge file. The filenames in the above map to the following:

signed.crt is the Let's Encrypt provided certificate
account.key is the user private key registered with LE
domain.csr is the cert request
domain.key is the key for the domain cert

Parsing WinMD with .NET reflection APIs

2016-11-02T00:00:00Z

Parsing WinMD files, the containers of WinRT API metadata, is relatively simple using the appropriate .NET reflection APIs. However, figuring out which reflection APIs to use is not obvious. I've got a completed C sharp class parsing WinMD files that you can check out for reference.

Use System.Reflection.Assembly.ReflectionOnlyLoad to load the WinMD file. Don't use the normal load methods because the WinMD files contain only metadata. This will load up info about APIs defined in that WinMD, but any references to types outside of that WinMD including types found in the normal OS system WinMD files must be resolved by the app code via the System.Reflection.InteropServices.WindowsRuntimeMetadata.ReflectionOnlyNamespaceResolve event.

In this event handler you must resolve the unknown namespace reference by adding an assembly to the NamespaceResolveEventArgs's ResolvedAssemblies property. If you're only interested in OS system WinMD files you can use System.Reflection.InteropServices.WindowsRuntimeMetadata.ResolveNamespace to turn a namespace into the expected OS system WinMD path and turn that path into an assembly with ReflectionOnlyLoad.

WPAD Server Fiddler Extension Source

2016-08-05T00:00:00Z

I've put my WPAD Fiddler extension source and the installer on GitHub.

Six years ago I made a WPAD DHCP server Fiddler extension (described previously and previously). The extension runs a WPAD DHCP server telling any clients that connect to connect to the running Fiddler instance. I've finally got around to putting the source on GitHub. I haven't touched it in five or so years so this is either for posterity or education or something.

Data breakpoints in JavaScript

2016-06-17T00:00:00Z

The other day I had to debug a JavaScript UWA that was failing when trying to use an undefined property. In a previous OS build this code would run and the property was defined. I wanted something similar to windbg/cdb's ba command that lets me set a breakpoint on read or writes to a memory location so I could see what was creating the object in the previous OS build and what that code was doing now in the current OS build. I couldn't find such a breakpoint mechanism in Visual Studio or F12 so I wrote a little script to approximate JavaScript data breakpoints.

The script creates a stub object with a getter and setter. It actually performs the get or set but also calls debugger; to break in the debugger. In order to handle my case of needing to break when window.object1.object2 was created or accessed, I further had it recursively set up such stub objects for the matching property names.

Its not perfect because it is an enumerable property and shows up in hasOwnProperty and likely other places. But for your average code that checks for the existence of a property via if (object.property) it works well.

WinRT Toast from PowerShell

2016-06-15T00:00:00Z

I've made a PowerShell script to show system toast notifications with WinRT and PowerShell. Along the way I learned several interesting things.

First off calling WinRT from PowerShell involves a strange syntax. If you want to use a class you write [-Class-,-Namespace-,ContentType=WindowsRuntime] first to tell PowerShell about the type. For example here I create a ToastNotification object:

[void][Windows.UI.Notifications.ToastNotification,Windows.UI.Notifications,ContentType=WindowsRuntime];
$toast = New-Object Windows.UI.Notifications.ToastNotification -ArgumentList $xml;

And here I call the static method CreateToastNotifier on the ToastNotificationManager class:

[void][Windows.UI.Notifications.ToastNotificationManager,Windows.UI.Notifications,ContentType=WindowsRuntime];
$notifier = [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier($AppUserModelId);

With this I can call WinRT methods and this is enough to show a toast but to handle the click requires a little more work.

To handle the user clicking on the toast I need to listen to the Activated event on the Toast object. However Register-ObjectEvent doesn't handle WinRT events. To work around this I created a .NET event wrapper class to turn the WinRT event into a .NET event that Register-ObjectEvent can handle. This is based on Keith Hill's blog post on calling WinRT async methods in PowerShell. With the event wrapper class I can run the following to subscribe to the event:

function WrapToastEvent {
    param($target, $eventName);

    Add-Type -Path (Join-Path $myPath "PoshWinRT.dll")
    $wrapper = new-object "PoshWinRT.EventWrapper[Windows.UI.Notifications.ToastNotification,System.Object]";
    $wrapper.Register($target, $eventName);
}

[void](Register-ObjectEvent -InputObject (WrapToastEvent $toast "Activated") -EventName FireEvent -Action { 
    ...
});

To handle the Activated event I want to put focus back on the PowerShell window that created the toast. To do this I need to call the Win32 function SetForegroundWindow. Doing so from PowerShell is surprisingly easy. First you must tell PowerShell about the function:

Add-Type @"
    using System;
    using System.Runtime.InteropServices;
    public class PInvoke {
        [DllImport("user32.dll")] [return: MarshalAs(UnmanagedType.Bool)]
        public static extern bool SetForegroundWindow(IntPtr hwnd);
    }
"@

Then to call:

[PInvoke]::SetForegroundWindow((Get-Process -id $myWindowPid).MainWindowHandle);

But figuring out the HWND to give to SetForegroundWindow isn't totally straight forward. Get-Process exposes a MainWindowHandle property but if you start a cmd.exe prompt and then run PowerShell inside of that, the PowerShell process has 0 for its MainWindowHandle property. We must follow up process parents until we find one with a MainWindowHandle:

$myWindowPid = $pid;
while ($myWindowPid -gt 0 -and (Get-Process -id $myWindowPid).MainWindowHandle -eq 0) {
    $myWindowPid = (gwmi Win32_Process -filter "processid = $($myWindowPid)" | select ParentProcessId).ParentProcessId;
}

Windows Store App WebView Cross Origin XMLHttpRequest Behavior

2016-06-02T00:00:00Z

TL;DR: Web content in a JavaScript Windows Store app or WebView in a Windows Store app that has full access to WinRT also gets to use XHR unrestricted by cross origin checks.

By default web content in a WebView control in a Windows Store App has the same sort of limitations as that web content in a web browser. However, if you give the URI of that web content full access to WinRT, then the web content also gains the ability to use XMLHttpRequest unrestricted by cross origin checks. This means no CORS checks and no OPTIONS requests. This only works if the web content's URI matches a Rule in the ApplicationContentUriRules of your app's manifest and that Rule declares WindowsRuntimeAccess="all". If it declares WinRT access as 'None' or 'AllowForWebOnly' then XHR acts as it normally does.

In terms of security, if you've already given a page access to all of WinRT which includes the HttpRequest class and other networking classes that don't perform cross origin checks, then allowing XHR to skip CORS doesn't make things worse.

WinRT Launcher API in PowerShell

2016-03-31T00:00:00Z

You can call WinRT APIs from PowerShell. Here's a short example using the WinRT Launcher API:

[Windows.System.Launcher,Windows.System,ContentType=WindowsRuntime]
$uri = New-Object System.Uri "http://example.com/"
[Windows.System.Launcher]::LaunchUriAsync($uri)

Note that like using WinRT in .NET, you use the System.Uri .NET class instead of the Windows.Foundation.Uri WinRT class which is not projected and under the covers the system will convert the System.Uri to a Windows.Foundation.Uri.

Cdb/Windbg Commands for Runtime Patching

2016-02-08T00:00:00Z

You can use conditional breakpoints and debugging commands in windbg and cdb that together can amount to effectively patching a binary at runtime. This can be useful if you have symbols but you can't easily rebuild the binary. Or if the patch is small and the binary requires a great deal of time to rebuild.

Skipping code

If you want to skip a chunk of code you can set a breakpoint at the start address of the code to skip and set the breakpoint's command to change the instruction pointer register to point to the address at the end of the code to skip and go. Voila you're skipping over that code now. For example:

bp 0x6dd6879b "r @eip=0x6dd687c3 ; g"

Changing parameters

You may want to modify parameters or variables and this is simple of course. In the following example a conditional breakpoint ANDs out a bit from dwFlags. Now when we run its as if no one is passing in that flag.

bp wiwi!RelativeCrack "?? dwFlags &= 0xFDFFFFFF;g"

Slightly more difficult is to modify string values. If the new string length is the same size or smaller than the previous, you may be able to modify the string value in place. But if the string is longer or the string memory isn't writable, you'll need a new chunk of memory into which to write your new string. You can use .dvalloc to allocate some memory and ezu to write a string into the newly allocated memory. In the following example I then overwrite the register containing the parameter I want to modify:

.dvalloc 100
ezu 000002a9`d4eb0000 "mfcore.dll"
r rcx = 000002a9`d4eb0000

Calling functions

You can also use .call to actually make new calls to methods or functions. Read more about that on the Old New Thing: Stupid debugger tricks: Calling functions and methods. Again, all of this can be used in a breakpoint command to effectively patch a binary.

Let's Encrypt NearlyFreeSpeech.net Setup

2016-02-04T00:00:00Z

2016-Nov-5: Updated post on using Let's Encrypt with NearlyFreeSpeech.net

I use NearlyFreeSpeech.net for my webhosting for my personal website and I've just finished setting up TLS via Let's Encrypt. The process was slightly more complicated than what you'd like from Let's Encrypt. So for those interested in doing the same on NearlyFreeSpeech.net, I've taken the following notes.

The standard Let's Encrypt client requires su/sudo access which is not available on NearlyFreeSpeech.net's servers. Additionally NFSN's webserver doesn't have any Let's Encrypt plugins installed. So I used the Let's Encrypt Without Sudo client. I followed the instructions listed on the tool's page with the addition of providing the "--file-based" parameter to sign_csr.py.

One thing the script doesn't produce is the chain file. But this topic "Let's Encrypt - Quick HOWTO for NSFN" covers how to obtain that:

curl -o domain.chn https://letsencrypt.org/certs/lets-encrypt-x1-cross-signed.pem

Now that you have all the required files, on your NFSN server make the directory /home/protected/ssl and copy your files into it. This is described in the NFSN topic provide certificates to NFSN. After copying the files and setting their permissions as described in the previous link you submit an assistance request. For me it was only 15 minutes later that everything was setup.

After enabling HTTPS I wanted to have all HTTP requests redirect to HTTPS. The normal Apache documentation on how to do this doesn't work on NFSN servers. Instead the NFSN FAQ describes it in "redirect http to https and HSTS". You use the X-Forwarded-Proto instead of the HTTPS variable because of how NFSN's virtual hosting is setup.

RewriteEngine on
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [L,R=301]

Turning on HSTS is as simple as adding the HSTS HTTP header. However, the description in the above link didn't work because my site's NFSN realm isn't on the latest Apache yet. Instead I added the following to my .htaccess. After I'm comfortable with everything working well for a few days I'll start turning up the max-age to the recommended minimum value of 180 days.

Header set Strict-Transport-Security "max-age=3600;"

Finally, to turn on CSP I started up Fiddler with my CSP Fiddler extension. It allows me to determine the most restrictive CSP rules I could apply and still have all resources on my page load. From there I found and removed inline script and some content loaded via http and otherwise continued tweaking my site and CSP rules.

After I was done I checked out my site on SSL Lab's SSL Test to see what I might have done wrong or needed improving. The first time I went through these steps I hadn't included the chain file which the SSL Test told me about. I was able to add that file to the same files I had already previously generated from the Let's Encrypt client and do another NFSN assistance request and 15 minutes later the SSL Test had upgraded me from 'B' to 'A'.

Unicode Clock

2016-01-24T00:00:00Z

I've made a Unicode Clock in JavaScript.

Unicode has code points for all 30 minute increments of clock faces. This is a simple project to display the one closest to the current time written in JavaScript.

Because the code points are all above 0xFFFF, I make use of some ES6 additions. I use the \u{XXXXXX} style escape sequence since the old style JavaScript escape sequence \uXXXX only supports code points up to 0xFFFF. I also use the method String.codePointAt rather than String.charCodeAt because the code points larger than 0xFFFF are represented in JavaScript strings using surrogate pairs and charCodeAt gives the surrogate value rather than codePointAt which gives the code point represented by the pair of surrogates.

"🕛".codePointAt(0)
128347
"🕛".charCodeAt(0)
55357

🕐🕑🕒🕓🕔🕕🕖🕗🕘🕙🕚🕛🕜🕝🕞🕟🕠🕡🕢🕣🕤🕥🕦🕧

The ordering of the code points does not make it simple to do this. I initially guessed the first code point in the range would be 12:00 followed by 12:30, 1:00 and so on. But actually 1:00 is first followed by all the on the hour times then all the half hour times.

JavaScript Types and WinRT Types

2016-01-21T00:00:00Z

Cloud Share - New App

2014-05-23T00:00:00Z

I've put a new app on the Windows Store: Cloud Share. It connects the web to your Windows 8 share charm.

I did the development on GitHub and quite enjoyed myself. I wasn't sure I liked the game-ification of development in GitHub's dashboard showing you your longest development streak in days. However I realized that it encourages me to do work on my personal project and anything that aids in holding my attention on and helping me finish these projects is a good thing.

location.hash and location.search are bad and they should feel bad

2014-05-22T00:00:00Z

The DOM location interface exposes the HTML document's URI parsed into its properties. However, it is ancient and has problems that bug me but otherwise rarely show up in the real world. Complaining about mostly theoretical issues is why blogging exists, so here goes:

The location object's search, hash, and protocol properties are all misnomers that lead to confusion about the correct terms:
- The 'search' property returns the URI's query property. The query property isn't limited to containing search terms.
- The 'hash' property returns the URI's fragment property. This one is just named after its delimiter. It should be called the fragment.
- The 'protocol' property returns the URI's scheme property. A URI's scheme isn't necessarily a protocol. The http URI scheme of course uses the HTTP protocol, but the https URI scheme is the HTTP protocol over SSL/TLS - there is no HTTPS protocol. Similarly for something like mailto - there is no mailto wire protocol.
The 'hash' and 'search' location properties both return null in the case that their corresponding URI property doesn't exist or if its the empty string. A URI with no query property and a URI with an empty string query property that are otherwise the same, are not equal URIs and are allowed by HTTP to return different content. Similarly for the fragment. Unless the specific URI scheme defines otherwise, an empty query or hash isn't the same as no query or hash.

But like complaining about the number of minutes in an hour none of this can ever change without huge compat issues on the web. Accordingly I can only give my thanks to Anne van Kesteren and the awesome work on the URL standard moving towards a more sane (but still working practically within the constraints of compat) location object and URI parsing in the browser.

Debugging anecdote - the color transparent black breaks accessibility

2014-05-22T00:00:00Z

Some time back while I was working on getting the Javascript Windows Store app platform running on Windows Phone (now available on the last Windows Phone release!) I had an interesting bug that in retrospect is amusing.

I had just finished a work item to get accessibility working for JS WinPhone apps when I got a new bug: With some set of JS apps, accessibility appeared to be totally broken. At that time in development the only mechanism we had to test accessibility was a test tool that runs on the PC, connects to the phone, and dumps out the accessibility tree of whatever app is running on the phone. In this bug, the tool would spin for a while and then timeout with an error and no accessibility information.

My first thought was this was an issue in my new accessibility code. However, debugging with breakpoints on my code I could see none of my code was run nor the code that should call it. The code that called that code was a more generic messaging system that hit my breakpoints constantly.

Rather than trying to work backward from the failure point, I decided to try and narrow down the repro and work forwards from there. One thing all the apps with the bug had in common was their usage of WinJS, but not all WinJS apps demonstrated the issue. Using a binary search approach on one such app I removed unrelated app code until all that was left was the app's usage of the WinJS AppBar and the bug still occurred. I replaced the WinJS AppBar usage with direct usage of the underlying AppBar WinRT APIs and continued.

Only some calls to the AppBar WinRT object produced the issue:

        var appBar = Windows.UI.WebUI.Core.WebUICommandBar.getForCurrentView(); 
        // appBar.opacity = 1;
        // appBar.closeDisplayMode = Windows.UI.WebUI.Core.WebUICommandBarClosedDisplayMode.default;
        appBar.backgroundColor = Windows.UI.Colors.white; // Bug!

Just setting the background color appeared to cause the issue and I didn't even have to display the AppBar. Through additional trial and error I was blown away to discover that some colors I would set caused the issue and other colors did not. Black wouldn't cause the issue but transparent black would. So would aqua but not white.

I eventually realized that predefined WinRT color values like Windows.UI.Colors.aqua would cause the issue while JS literal based colors didn't cause the issue (Windows.UI.Color is a WinRT struct which projects in JS as a JS literal object with the struct members as JS object properties so its easy to write something like {r: 0, g: 0, b: 0, a: 0} to make a color) and I had been mixing both in my tests without realizing there would be a difference. I debugged into the backgroundColor property setter that consumed the WinRT color struct to see what was different between Windows.UI.Colors.black and {a: 1, r: 0, g: 0, b: 0} and found the two structs to be byte wise exactly the same.

On a hunch I tried my test app with only a reference to the color and otherwise no interaction with the AppBar and not doing anything with the actual reference to the color: Windows.UI.Colors.black;. This too caused the issue. I knew that the implementation for these WinRT const values live in a DLL and guessed that something in the code to create these predefined colors was causing the issue. I debugged in and no luck. Now I also have experienced crusty code that would do exciting things in its DllMain, the function that's called when a DLL is loaded into the process so I tried modifying my C++ code to simply LoadLibrary the DLL containing the WinRT color definition, windows.ui.xaml.dll and found the bug still occurred! A short lived moment of relief as the world seemed to make sense again.

Debugging into DllMain nothing interesting happened. There were interesting calls in there to be sure, but all of them behind conditions that were false. I was again stumped. On another hunch I tried renaming the DLL and only LoadLibrary'ing it and the bug went away. I took a different DLL renamed it windows.ui.xaml.dll and tried LoadLibrary'ing that and the bug came back. Just the name of the DLL was causing the issue.

I searched for the DLL name in our source code index and found hits in the accessibility tool. Grinning I opened the source to find that the accessibility tool's phone side service was trying to determine if a process belonged to a XAML app or not because XAML apps had a different accessibility contract. It did this by checking to see if windows.ui.xaml.dll was loaded in the target process.

At this point I got to fix my main issue and open several new bugs for the variety of problems I had just run into. This is a how to on writing software that is difficult to debug.

FitBit and WebOC Application Compatibility Errors

2013-08-29T00:00:00Z

I just got a FitBit One from my wife. Unfortunately I had issues running their app on my Windows 8.1 Preview machine. But I recognized the errors as IE compatibility issues, for instance an IE dialog popup from the FitBit app telling me about an error in the app's JavaScript. Given my previous post on WebOC versioning you may guess what I tried next. I went into the registry and tried out different browser mode and document mode versions until I got the FitBit software running without error. Ultimately I found the following registry value to work well ('FitBit connect.exe' set to DWORD decimal 8888).

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION]
"Fitbit Connect.exe"=dword:000022b8

For those familiar with the Windows registry the above should be enough. For those not familiar, copy and paste the above into notepad, save as a file named "fitbit.reg", and then double click the reg file and say 'Yes' to the prompt. Hopefully in the final release of Windows 8.1 this won't be an issue.

Moving PowerShell data into Excel

2013-08-15T00:00:00Z

PowerShell nicely includes ConvertTo-CSV and ConvertFrom-CSV which allow you to serialize and deserialize your PowerShell objects to and from CSV. Unfortunately the CSV produced by ConvertTo-CSV is not easily opened by Excel which expects by default different sets of delimiters and such. Looking online you'll find folks who recommend using automation via COM to create a new Excel instance and copy over the data in that fashion. This turns out to be very slow and impractical if you have large sets of data. However you can use automation to open CSV files with not the default set of delimiters. So the following isn't the best but it gets Excel to open a CSV file produced via ConvertTo-CSV and is faster than the other options:

Param([Parameter(Mandatory=$true)][string]$Path);

$excel = New-Object -ComObject Excel.Application

$xlWindows=2
$xlDelimited=1 # 1 = delimited, 2 = fixed width
$xlTextQualifierDoubleQuote=1 # 1= doublt quote, -4142 = no delim, 2 = single quote
$consequitiveDelim = $False;
$tabDelim = $False;
$semicolonDelim = $False;
$commaDelim = $True;
$StartRow=1
$Semicolon=$True

$excel.visible=$true
$excel.workbooks.OpenText($Path,$xlWindows,$StartRow,$xlDelimited,$xlTextQualifierDoubleQuote,$consequitiveDelim,$tabDelim,$semicolonDelim, $commaDelim);

See Workbooks.OpenText documentation for more information.

Serializing JavaScript Promise Execution

2013-08-10T00:00:00Z

Occasionally I have need to run a set of unrelated promises in series, for instance an object dealing with a WinRT camera API that can only execute one async operation at a time, or an object handling postMessage message events and must resolve associated async operations in the same order it received the requests. The solution is very simply to keep track of the last promise and when adding a new promise in serial add a continuation of the last promise to execute the new promise and point the last promise at the result. I encapsulate the simple solution in a simple constructor:

    function PromiseExecutionSerializer() {
        var lastPromise = WinJS.Promise.wrap(); // Start with an empty fulfilled promise.

        this.addPromiseForSerializedExecution = function(promiseFunction) {
            lastPromise = lastPromise.then(function () {
                // Don't call directly so next promise doesn't get previous result parameter.
                return promiseFunction();
            });
        }
    }

The only thing to watch out for is to ensure you don't pass the result of a previous promise onto a subsequent promise that is unrelated.

Considerate MessagePort Usage

2013-08-07T00:00:00Z

When writing a JavaScript library that uses postMessage and the message event, I must be considerate of other JS code that will be running along side my library. I shouldn't assume I'm the only sender and receiver on a caller provided MessagePort object. This means obviously I should use addEventListener("message" rather than the onmessage property (see related What if two programs did this?). But considering the actual messages traveling over the message channel I have the issue of accidentally processing another libraries messages and having another library accidentally process my own message. I have a few options for playing nice in this regard:

Require a caller provided unique MessagePort: This solves the problem but puts a lot of work on the caller who may not notice nor follow this requirement.
Uniquely mark my messages: To ensure I'm acting upon my own messages and not messages that happen to have similar properties as my own, I place a 'type' property on my postMessage data with a value of a URN unique to me and my JS library. Usually because its easy I use a UUID URN. There's no way someone will coincidentally produce this same URN. With this I can be sure I'm not processing someone else's messages. Of course there's no way to modify my postMessage data to prevent another library from accidentally processing my messages as their own. I can only hope they take similar steps as this and see that my messages are not their own.
Use caller provided MessagePort only to upgrade to new unique MessagePort: I can also make my own unique MessagePort for which only my library will have the end points. This does still require the caller to provide an initial message channel over which I can communicate my new unique MessagePort which means I still have the problems above. However it clearly reduces the surface area of the problem since I only need once message to communicate the new MessagePort.

The best solution is likely all of the above.
Photo is Sharing by leezie5. Two squirrels sharing food hanging from a bird feeder. Used under Creative Commons license Attribution-NonCommercial-NoDerivs 2.0 Generic.

URI functions in Windows Store Applications

2013-07-25T00:00:00Z

C++ constructor member initializers run in member declaration order

2013-07-18T00:00:00Z

Subtleties of postMessage

2013-07-15T00:00:00Z

In IE10 and other new browsers one may create MessageChannel objects that have two MessagePorts each connected (w3c spec calls it entangled) to one another such that postMessage on one port results in the message event firing on the other. You can pass an array of ports as the last parameter to postMessage and they show up in the ports property of the message event arg.

Origin

The postMessage here is like the worker postMessage and unlike the window and iframe postMessage in that it applies no origin checking:

No origin postMessage in workers and MessagePorts: postMessage(messageData, ports)
Origin postMessage in windows and iframes: postMessage(messageData, targetOrigin, ports)

Unfortunately the origin isn't an optional parameter at the end to make the two postMessages have the same signature.

On the event handler side, the event arg always has an origin property. But in the no origin case it is always the empty string.

Source

There is also a source property on the message event arg which if set is an object that has a postMessage property allowing you to post back to your caller. It is set for the origin case, however, in the no origin case this property is null. This is somewhat reasonable because in the case of MessagePort and Workers there are only two endpoints so you always know the source of a message implicitly. Unlike the origin case in which any iframe or window can be calling postMessage on any other iframe or window and the caller is unknown. So not unreasonable but it would be nice if the source property was always set for consistency.

MessageChannel start

When a MessageChannel is created it has two MessagePorts, but until those ports are started they will queue up any messages they receive. Once started they will dispatch all queued messages. Ports don't have to be started to send messages.

A port may be started in two ways, either by explicitly calling the start method on the port, or by setting the onmessage callback property on the port. However, adding an event listener via addEventListener("message", does not start the port. It works this way in IE and Chrome and the spec states this as well.

The justification is that since you can have only one callback via onmessage that once set you must implicitly be ready to receive messages and its fine to start the port. As opposed to the addEventListener in which case the user agent cannot start implicitly because it doesn't know how many event listeners will be added. I found Hixie stating this justification in geoloc meeting notes.

Links

W3C Spec

Opera introduction

Percent Clcok Windows Store App Development Notes

2013-07-11T00:00:00Z

My third completed Windows Store app is Percent Clock which displays portions of a time span like the time of the day or time until your next birthday, as a percentage. This was a small project I had previously started as a webpage and converted and finished as an HTML JavaScript Windows Store app.

The only somewhat interesting aspect of this app is that its the first app for which I tried charging. I picked the minimum amount for price 1.49 USD as it is a simple app and unsurprisingly it has sold very poorly. I'm considering releasing new instances of the app for specific scenarios:

Death Clock: viewing your current age with respect to your life expectancy as a percentage.
New Year Countdown: percentage of the year until New Years.

WinRT PropertySet Changed Event Danger

2013-07-08T00:00:00Z

The Windows Runtime API Windows.Foundation.Collections.PropertySet class is a nice string name to object value map that has a changed event that fires when the contents of the map is modified. Be careful with this event because it fires synchronously from the thread on which the PropertySet was modified. If modified from the UI thread, the UI thread will then wait as it synchronously dispatches the changed event to all listeners which could lead to performance issues or especially from the UI thread deadlock. For instance, deadlock if you have two threads both trying to tell each other about changed events for different PropertySets.

Words with Hints Windows 8 App Development Notes

2013-07-04T00:00:00Z

My second completed app for the Windows Store was Words with Hints a companion to Words with Friends or other Scrabble like games that gives you *ahem* hints. You provide your tiles and optionally letters placed in a line on the board and Words with Hints gives you word options.

I wrote this the first time by building a regular expression to check against my dictionary of words which made for a slow app on the Surface. In subsequent release of the app I now spawn four web workers (one for each of the Surface's cores) each with its own fourth of my dictionary. Each fourth of the dictionary is a trie which makes it easy for me to discard whole chunks of possible combinations of Scrabble letters as I walk the tree of possibilities.

The dictionaries are large and takes a noticeable amount of time to load on the Surface. The best performing mechanism I found to load them is as JavaScript source files that simply define their portion of the dictionary on the global object and synchronously (only on the worker so not blocking the UI thread). Putting them into .js files means they take advantage of bytecode caching making them load faster. However because the data is mostly strings and not code there is a dramatic size increase when the app is installed. The total size of the four dictionary .js files is about 44Mb. The bytecode cache for the dictionary files is about double that 88Mb meaning the dictionary plus the bytecode cache is 132Mb.

To handle the bother of postMessage communication and web workers this was the first app in which I used my promise MessagePort project which I'll discuss more in the future.

This is the first app in which I used the Microsoft Ad SDK. It was difficult to find the install for the SDK and difficult to use their website, but once setup, the Ad SDK was easy to import into VS and easy to use in my app.

MSVC++ 64bit Enums

2013-07-01T00:00:00Z

If you want to represent a value larger than 32bits in an enum in MSVC++ you can use C++0x style syntax to tell the compiler exactly what kind of integral type to store the enum values. Unfortunately by default an enum is always 32bits, and additionally while you can specify constants larger than 32bits for the enum values, they are silently truncated to 32bits.

For instance the following doesn't compile because Lorem::a and Lorem::b have the same value of '1':


enum Lorem {
    a = 0x1,
    b = 0x100000001
} val;

switch (val) {
    case Lorem::a:
        break;
    case Lorem::b:
        break;
}

Unfortunately it is not an error to have b's constant truncated, and the previous without the switch statement does compile just fine:


enum Lorem {
    a = 0x1,
    b = 0x100000001
} val;

But you can explicitly specify that the enum should be represented by a 64bit value and get expected compiling behavior with the following:


enum Lorem : UINT64 {
    a = 0x1,
    b = 0x100000001
} val;

switch (val) {
    case Lorem::a:
        break;
    case Lorem::b:
        break;
}

Shout Text Windows 8 App Development Notes

2013-06-27T00:00:00Z

My first app for Windows 8 was Shout Text. You type into Shout Text, and your text is scaled up as large as possible while still fitting on the screen, as you type. It is the closest thing to a Hello World app as you'll find on the Windows Store that doesn't contain that phrase (by default) and I approached it as the simplest app I could make to learn about Windows modern app development and Windows Store app submission.

I rely on WinJS's default layout to use CSS transforms to scale up the user's text as they type. And they are typing into a simple content editable div.

The app was too simple for me to even consider using ads or charging for it which I learned more about in future apps.

The first interesting issue I ran into was that copying from and then pasting into the content editable div resulted in duplicates of the containing div with copied CSS appearing recursively inside of the content editable div. To fix this I had to catch the paste operation and remove the HTML data from the clipboard to ensure only the plain text data is pasted:

        function onPaste() {
            var text;

            if (window.clipboardData) {
                text = window.clipboardData.getData("Text").toString();
                window.clipboardData.clearData("Html");
                window.clipboardData.setData("Text", util.normalizeContentEditableText(text));
            }
        }
        shoutText.addEventListener("beforepaste", function () { return false; }, false);
        shoutText.addEventListener("paste", onPaste, false);

I additionally found an issue in IE in which applying a CSS transform to a content editable div that has focus doesn't move the screen position of the user input caret - the text is scaled up or down but the caret remains the same size and in the same place on the screen. To fix this I made the following hack to reapply the current cursor position and text selection which resets the screen position of the user input caret.

        function resetCaret() {
            setTimeout(function () {
                var cursorPos = document.selection.createRange().duplicate();
                cursorPos.select();
            }, 200);
        }

        shoutText.attachEvent("onresize", function () { resetCaret(); }, true);

Windows Store on Windows 8 Fun For Independent Developers

2013-06-24T00:00:00Z

Having worked on Windows 8 I'm not in a neutral position to review aspects of it, however I'll say from a high level I love taking the following various positives from smart phone apps and app stores and applying it to the desktop:

Independent developers can easily publish apps.
One trusted place for a user to find apps.
User can trust apps are limited to a declared set of capabilities.
One common and easy way for users to buy and try apps.
Easy mechanism for independent developers to collect revenue.

Relieving the independent developer of software development overhead, in this case Windows taking care of distribution and sales infrastructure is wonderful for me with my third party developer hat on. This combined with my new found fun of developing in JavaScript and the new Windows Runtime APIs means I've been implementing and finishing various ideas I've had - some for fun and some for productivity on my Surface. Development notes to follow.

Sci-fi short stories disguised as Internet docs

2013-05-29T00:00:00Z

The recent short story Twitter API returning results that do not respect arrow of time by Tim May written as a Twitter bug report reminded me of a few other short sci-fi stories written in the style of some sort of Internet document:

comp.basalisk FAQ by David Langford written as a FAQ.
Wikihistory by Desmond Warzel presented as a forum thread.
The Riddle of the Universe and Its Solution by Christopher Cherian written as a scientific report.

Windows Remote Desktop via Internet

2012-12-07T00:00:00Z

To setup my home Windows dev box to be accessible from outside I followed two main steps:

Configure my router to forward the remote desktop port.
Setup a free dynamic DNS service.

Last time I had to do this there was a service named dynamicdns.org which seems to still exist but no longer appears to be free. Instead I used dnsdynamic.org which is free and has a web API as well as links to and instructions for setting up native tools to dynamically update my IP address.

Stripe CTF - Level 8

2012-12-07T00:00:00Z

Level 8 of the Stripe CTF is a password server that returns success: true if and only if the password provided matches the password stored directly via a RESTful API and optionally indirectly via a callback URI. The solution is side channel attack like a timing attack but with ports instead of time.

(I found this in my drafts folder and had intended to post a while ago.)

Code

    def nextServerCallback(self, data):
        parsed_data = json.loads(data)
        # Chunk was wrong!
        if not parsed_data['success']:
            # Defend against timing attacks
            remaining_time = self.expectedRemainingTime()
            self.log_info('Going to wait %s seconds before responding' %
                          remaining_time)
            reactor.callLater(remaining_time, self.sendResult, False)
            return

        self.checkNext()

Issue

The password server breaks the target password into four pieces and stores each on a different server. When a password request is sent to the main server it makes requests to the sub-servers for each part of the password request. It does this in series and if any part fails, then it stops midway through. Password requests may also be made with corresponding URI callbacks and after the server decides on the password makes an HTTP request on the provided URI callbacks saying if the password was success: true or false.
A timing attack looks at how long it took for a password to be rejected and longer times could mean a longer prefix of the password was correct allowing for a directed brute force attack. Timing attacks are prevented in this case by code on the password server that attempts to wait the same amount of time, even if the first sub-server responds with false. However, the server uses sequential outgoing port numbers shared between the requests to the sub-servers and the callback URIs. Accordingly, we can examine the port numbers on our callback URIs to direct a brute force attack.
If the password provided is totally incorrect then the password server will contact one sub-server and then your callback URI. So if you see the remote server's port number go up by two when requesting your callback URI, you know the password is totally incorrect. If by three then you know the first fourth of the password is correct and the rest is incorrect. If by four then two fourths of the password is correct. If by five then four sub-servers were contacted so you need to rely on the actual content of the callback URI request of 'success: true' or 'false' since you can't tell from the port change if the password was totally correct or not.
The trick in the real world is false positives. The port numbers are sequential over the system, so if the password server is the only thing making outgoing requests then its port numbers will also be sequential, however other things on the system can interrupt this. This means that the password server could contact three sub-servers and normally you'd see the port number increase by four, but really it could increase by four or more because of other things running on the system. To counteract this I ran in cycles: brute forcing the first fourth of the password and removing any entry that gets a two port increase and keeping all others. Eventually I could remove all but the correct first fourth of the password. And so on for the next parts of the password.
I wrote my app to brute force this in Python. This was my first time writing Python code so it is not pretty.

Stripe CTF - Level 7

2012-09-13T00:00:00Z

Stripe CTF - Level 5

2012-09-11T00:00:00Z

Level 5 of the Stripe CTF revolved around a design issue in an OpenID like protocol.

Code

    def authenticated?(body)
      body =~ /[^\w]AUTHENTICATED[^\w]*$/
    end

...

      if authenticated?(body)
        session[:auth_user] = username
        session[:auth_host] = host
        return "Remote server responded with: #{body}." \
               " Authenticated as #{username}@#{host}!"

Issue

This level is an implementation of a federated identity protocol. You give it an endpoint URI and a username and password, it posts the username and password to the endpoint URI, and if the response is 'AUTHENTICATED' then access is allowed. It is easy to be authenticated on a server you control, but this level requires you to authenticate from the server running the level. This level only talks to stripe CTF servers so the first step is to upload a document to the level 2 server containing the text 'AUTHENTICATED' and we can now authenticate on a level 2 server. Notice that the level 5 server will dump out the content of the endpoint URI and that the regexp it uses to detect the text 'AUTHENTICATED' can match on that dump. Accordingly I uploaded an authenticated file to

https://level02-2.stripe-ctf.com/user-ajvivlehdt/uploads/authenticated

Using that as my endpoint URI means authenticating as level 2. I can then choose the following endpoint URI to authenticate as level 5.

https://level05-1.stripe-ctf.com/user-qtoyekwrod/?pingback=https%3A%2F%2Flevel02-2.stripe-ctf.com%2Fuser-ajvivlehdt%2Fuploads%2Fauthenticated&username=a&password=a

Navigating to that URI results in the level 5 server telling me I'm authenticated as level 2 and lists the text of the level 2 file 'AUTHENTICATED'. Feeding this back into the level 5 server as my endpoint URI means level 5 seeing 'AUTHENTICATED' coming back from a level 5 URI.

Notes

I didn't see any particular code review red flags, really the issue here is that the regular expression testing for 'AUTHENTICATED' is too permisive and the protocol itself doesn't do enough. The protocol requires only a set piece of common literal text to be returned which makes it easy for a server to accidentally fall into authenticating. Having the endpoint URI have to return variable text based on the input would make it much harder for a server to accidentally authenticate.

Stripe CTF - XSS, CSRF (Levels 4 & 6)

2012-09-10T00:00:00Z

Level 4 and level 6 of the Stripe CTF had solutions around XSS.

Level 4

Code

> Registered Users

<%= user[:username] %> (password: <%= user[:password] %>, last active <%= last_active %>)

Issue

The level 4 web application lets you transfer karma to another user and in doing so you are also forced to expose your password to that user. The main user page displays a list of users who have transfered karma to you along with their password. The password is not HTML encoded so we can inject HTML into that user's browser. For instance, we could create an account with the following HTML as the password which will result in XSS with that HTML:

Notes

Code review red flags in this case included lack of encoding when using user controlled content to create HTML content, storing passwords in plain text in the database, and displaying passwords generally. By design the web app shows users passwords which is a very bad idea.

Level 6

Code



...

    def self.safe_insert(table, key_values)
      key_values.each do |key, value|
        # Just in case people try to exfiltrate
        # level07-password-holder's password
        if value.kind_of?(String) &&
            (value.include?('"') || value.include?("'"))
          raise "Value has unsafe characters"
        end
      end

      conn[table].insert(key_values)
    end

Issue

This web app does a much better job than the level 4 app with HTML injection. They use encoding whenever creating HTML using user controlled data, however they don't use encoding when injecting JSON data into script (see post_data initialization above). This JSON data is the last five most recent messages sent on the app so we get to inject script directly. However, the system also ensures that no strings we write contains single or double quotes so we can't get out of the string in the JSON data directly. As it turns out, HTML lets you jump out of a script block using no matter where you are in script. For instance, in the middle of a value in some JSON data we can jump out of script. But we still want to run script, so we can jump right back in. So the frame so far for the message we're going to post is the following:

Stripe CTF - Input validation (Levels 1 & 2)

2012-09-06T00:00:00Z

Stripe's web security CTF's Level 1 and level 2 of the Stripe CTF had issues with missing input validation solutions described below.

Level 1

Code

    <?php
      $filename = 'secret-combination.txt';
      extract($_GET);
      if (isset($attempt)) {
        $combination = trim(file_get_contents($filename));
        if ($attempt === $combination) {

Issue

The issue here is the usage of the extract php method which extracts name value pairs from the map input parameter and creates corresponding local variables. However this code uses $_GET which contains a map of name value pairs passed in the query of the URI. The expected behavior is to get an attempt variable out, but since no input validation is done I can provide a filename variable and overwrite the value of $filename. Providing an empty string gives an empty string $combination which I can match with an empty string $attempt. So without knowing the combination I can get past the combination check.

Notes

Code review red flag in this case was the direct use of $_GET with no validation. Instead of using extract the developer could try to extract specifically the attempt variable manually without using extract.

Level 2

Code

    $dest_dir = "uploads/";
    $dest = $dest_dir . basename($_FILES["dispic"]["name"]);
    $src = $_FILES["dispic"]["tmp_name"];
    if (move_uploaded_file($src, $dest)) {
      $_SESSION["dispic_url"] = $dest;
      chmod($dest, 0644);
      echo "Successfully uploaded your display picture.
";
    }

Issue

This code accepts POST uploads of images but with no validation to ensure it is not an arbitrary file. And even though it uses chmod to ensure the file is not executable, things like PHP don't require a file to be executable in order to run them. Accordingly, one can upload a PHP script, then navigate to that script to run it. My PHP script dumped out the contents of the file we're interested in for this level:

Notes

Code review red flags include manual file management, chmod, and use of file and filename inputs without any kind of validation. If this code controlled the filename and ensured that the extension was one of a set of image extensions, this would solve this issue. Due to browser mime sniffing its additionally a good idea to serve a content-type that starts with "image/" for these uploads to ensure browsers treat these as images and not sniff for script or HTML.

Stripe CTF - SQL injections (Levels 0 & 3)

2012-09-05T00:00:00Z

Stripe's web security CTF's level 0 and level 3 had SQL injection solutions described below.

Level 0

Code

app.get('/*', function(req, res) {
  var namespace = req.param('namespace');

  if (namespace) {
    var query = 'SELECT * FROM secrets WHERE key LIKE ? || ".%"';
    db.all(query, namespace, function(err, secrets) {

Issue

There's no input validation on the namespace parameter and it is injected into the SQL query with no encoding applied. This means you can use the '%' character as the namespace which is the wildcard character matching all secrets.

Notes

Code review red flag was using strings to query the database. Additional levels made this harder to exploit by using an API with objects to construct a query rather than strings and by running a query that only returned a single row, only ran a single command, and didn't just dump out the results of the query to the caller.

Level 3

Code

@app.route('/login', methods=['POST'])
def login():
    username = flask.request.form.get('username')
    password = flask.request.form.get('password')

    if not username:
        return "Must provide username\n"

    if not password:
        return "Must provide password\n"

    conn = sqlite3.connect(os.path.join(data_dir, 'users.db'))
    cursor = conn.cursor()

    query = """SELECT id, password_hash, salt FROM users
               WHERE username = '{0}' LIMIT 1""".format(username)
    cursor.execute(query)

    res = cursor.fetchone()
    if not res:
        return "There's no such user {0}!\n".format(username)
    user_id, password_hash, salt = res

    calculated_hash = hashlib.sha256(password + salt)
    if calculated_hash.hexdigest() != password_hash:
        return "That's not the password for {0}!\n".format(username)

Issue

There's little input validation on username before it is used to constrcut a SQL query. There's no encoding applied when constructing the SQL query string which is used to, given a username, produce the hashed password and the associated salt. Accordingly one can make username a part of a SQL query command which ensures the original select returns nothing and provide a new SELECT via a UNION that returns some literal values for the hash and salt. For instance the following in blue is the query template and the red is the username injected SQL code:

SELECT id, password_hash, salt FROM users WHERE username = 'doesntexist' UNION SELECT id, ('5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8') AS password_hash, ('word') AS salt FROM users WHERE username = 'bob' LIMIT 1

In the above I've supplied my own salt and hash such that my salt (word) plus my password (pass) hashed produce the hash I provided above. Accordingly, by providing the above long and interesting looking username and password as 'pass' I can login as any user.

Notes

Code review red flag is again using strings to query the database. Although this level was made more difficult by using an API that returns only a single row and by using the execute method which only runs one command. I was forced to (as a SQL noob) learn the syntax of SELECT in order to figure out UNION and how to return my own literal values.

Stripe Web Security CTF Summary

2012-08-30T00:00:00Z

I was the 546th person to complete Stripe's web security CTF and again had a ton of fun applying my theoretical knowledge of web security issues to the (semi-)real world. As I went through the levels I thought about what red flags jumped out at me (or should have) that I could apply to future code reviews:

Level	Issue	Code Review Red Flags
0	Simple SQL injection	No encoding when constructing SQL command strings. Constructing SQL command strings instead of SQL API
1	extract($_GET);	No input validation.
2	Arbitrary PHP execution	No input validation. Allow file uploads. File permissions modification.
3	Advanced SQL injection	Constructing SQL command strings instead of SQL API.
4	HTML injection, XSS and CSRF	No encoding when constructing HTML. No CSRF counter measures. Passwords stored in plain text. Password displayed on site.
5	Pingback server doesn't need to opt-in	n/a - By design protocol issue.
6	Script injection and XSS	No encoding while constructing script. Deny list (of dangerous characters). Passwords stored in plain text. Password displayed on site.
7	Length extension attack	Custom crypto code. Constructing SQL command string instead of SQL API.
8	Side channel attack	Password handling code. Timing attack mitigation too clever.

More about each level in the future.

Web Security Contest - Stripe CTF

2012-08-27T00:00:00Z

Stripe is running a web security capture the flag - a series of increasingly difficult web security exploit challenges. I've finished it and had a lot of fun. Working on a web browser I knew the theory of these various web based attacks, but this was my first chance to put theory into practice with:

No adverse consequences
Knowledge that there is a fun security exploit to find
Access to the server side source code

Here's a blog post on the CTF behind the scenes setup which has many impressive features including phantom users that can be XSS/CSRF'ed.

I'll have another post on my difficulties and answers for the CTF levels after the contest is over on Wed, but if you're looking for hints, try out the CTF chatroom or the level specific CTF chatroom.

HTTP Compression Documentation Reference

2012-06-13T00:00:00Z

There's a lot of name reuse in HTTP compression so I've made the following to help myself keep it straight.

HTTP Content Coding Token	gzip	deflate	compress
	An encoding format produced by the file compression program "gzip" (GNU zip)	The "zlib" format as described in RFC 1950.	The encoding format produced by the common UNIX file compression program "compress".
Data Format	GZIP file format	ZLIB Compressed Data Format	The compress program's file format
Compression Method	Deflate compression method		LZW
	Deflate consists of LZ77 and Huffman coding

Compress doesn't seem to be supported by popular current browsers, possibly due to its past with patents.

Deflate isn't done correctly all the time. Some servers would send the deflate data format instead of the zlib data format and at least some versions of Internet Explorer expect deflate data format instead of zlib data format.

Application Protocols in Windows 8

2012-06-12T00:00:00Z

In Windows 8 you can still register a desktop application to handle a particular URI scheme, but now you can also register a Metro Win8 application to handle a particular URI scheme. No more manually modifying the registry - now there's pretty UI in VS to handle this.

Changing Windows Live IDs

2012-06-06T00:00:00Z

Use of my old Hotmail account has really snuck up on me as I end up caring more and more about all of the services with which it is associated. The last straw is Windows 8 login, but previous straws include Xbox, Zune, SkyDrive, and my Windows 7 Phone. I like the features and sync'ing associated with the Windows Live ID, but I don't like my old, spam filled, hotmail email address on the Live ID account.

A coworker told me about creating a Live ID from a custom domain, which sounded like just the ticket for me. Following the instructions above I was able to create a new deletethis.net Live ID but the next step of actually using this new Live ID was much more difficult. My first hope was there would be some way to link my new and old Live IDs so as to make them interchangeable. As it turns out there is a way to link Live IDs but all that does is make it easy to switch between accounts on Live Mail, SkyDrive and some other webpages.

Instead one must change over each service or start over depending on the service:

Xbox: In the Xbox 360 system menu you can change the Live ID associated with your gamertag. This worked fine for me and I got an email telling me about the transfer of my Microsoft Points.
Zune: There's no way to do this for the Zune specifically, however changing over your Xbox account also transfers over all your Zune purchased content. I don't have a Zune Pass so I can't confirm that, but all of my previously purchased television shows transferred over successfully.
Windows 7 Phone: To change the main Live ID associated with your phone, reset your phone to factory default and start over. All purchased applications are lost. Had I purchased any applications I would have been pissed, but instead I was just irritated that I had to reset my phone.
Mail: I don't use my Hotmail account for anything and it only sits and collects spam. Accordingly I didn't attempt switching this over.
SkyDrive: I didn't have much in my SkyDrive account. I downloaded all files as a zip and then manually uploaded them to the new account.

Crowdsource These Projects

2012-05-22T00:00:00Z

I keep seeing crowdsource projects with big names that I actually want to back:

Permanently Add Path to System PATH Environment Variable in PowerShell

2012-05-17T00:00:00Z

According to MSDN the proper way to permanently add a path to your system's PATH environment variable is by modifying a registry value. Accordingly this is easily represented in a PowerShell script that first checks if the path provided is already there and otherwise appends it:

param([Parameter(Mandatory = $true)] [string] $Path);
$FullPathOriginal = (gp "HKLM:\System\CurrentControlSet\Control\Session Manager\Environment").Path;
if (!($FullPathOriginal.split(";") | ?{ $_ -like $Path })) {
    sp "HKLM:\System\CurrentControlSet\Control\Session Manager\Environment" -name Path -value ($FullPathOriginal + ";" +
 $Path);
}

PowerShell Equivalents for JavaScript Array Functions

2012-05-15T00:00:00Z

Built-in

map: input | %{ expression($_) }
forEach: input | %{ [void]expression($_) }
filter: input | ?{ expression($_) }
indexOf: input.indexOf(value)

Close to built-in

some: if (input | ?{ expression($_) }) { ... }
every: if (-not input | ?{ !expression($_) }) { ... }
lastIndexOf: [array]::lastIndexOf(input, value)

Write it yourself

reduce: function reduce($fn, $a, $init) { $s = $init; $a | %{ $s = &$fn $s $_; }; $s; }

Favorite Windows 8 Feature: Intra-Line Tab Completion

2012-05-09T00:00:00Z

Fixed in Windows 8 is intra-line tab completion - you can try it out on the Windows 8 Consumer Preview now. If you open a command prompt, type a command, then move your cursor back into a token in the middle of the command and tab complete, the tab completion works on that whitespace delimited token and doesn't erase all text following the cursor. Like it does in pre Windows 8. And annoys the hell out of me. Yay!

Play Fez

2012-05-07T00:00:00Z

I'm done playing Fez. The style is atmospheric retro nastalgia and on the surface the gameplay is standard 2D platformer with one interesting Flatland style game mechanic but dig deeper to find Myst style puzzles. Despite the following I thoroughly enjoyed the game and would recommend it to anyone intrigued by the previous. Five eighths through the game I ran into one of the game's infamous Fez save game breaking issues, but I enjoyed the game enough that I started over before any patches were released and had no further issues.

While playing the game I created some tools to help keep track of my Fez notes (spoilers) including a Pixelated Image Creator that makes it easy to generate data URIs for large, black and white pixelated images, and (spoilers) a Fez Transliteration Tool to help me translate the in-game writing system.

Why Did This Work?

2012-03-23T00:00:00Z

Do we have a word or phrase to describe the following situation: You code up something complicated and it compiles and works on the first try. You then spend the next ten minutes trying to figure out what's actually broken because it shouldn't be this easy.

Or in meme form:

Alternate IPv4 Forms - URI Host Syntax Notes

2012-03-14T00:00:00Z

By the URI RFC there is only one way to represent a particular IPv4 address in the host of a URI. This is the standard dotted decimal notation of four bytes in decimal with no leading zeroes delimited by periods. And no leading zeros are allowed which means there's only one textual representation of a particular IPv4 address.

However as discussed in the URI RFC, there are other forms of IPv4 addresses that although not officially allowed are generally accepted. Many implementations used inet_aton to parse the address from the URI which accepts more than just dotted decimal. Instead of dotted decimal, each dot delimited part can be in decimal, octal (if preceded by a '0') or hex (if preceded by '0x' or '0X'). And that's each section individually - they don't have to match. And there need not be 4 parts: there can be between 1 and 4 (inclusive). In case of less than 4, the last part in the string represents all of the left over bytes, not just one.

For example the following are all equivalent:

192.168.1.1: Standard dotted decimal form
0300.0250.01.01: Octal
0xC0.0XA8.0x1.0X1: Hex
192.168.257: Fewer parts
0300.0XA8.257: All of the above

The bread and butter of URI related security issues is when one part of the system disagrees with another about the interpretation of the URI. So this non-standard, non-normal form syntax has been been a great source of security issues in the past. Its mostly well known now (CreateUri normalizes these non-normal forms to dotted decimal), but occasionally a good tool for bypassing naive URI blocking systems.

Privacy through Obscurity

2012-03-09T00:00:00Z

With Facebook changing its privacy policy and settings so frequently and just generally the huge amount of social sites out there, for many of us it is far too late to ensure our name doesn't show up with unfortunate results in web searches. Information is too easily copyable and archive-able to make removing these results a viable option, so clearly the solution is to create more data.

Create fake profiles on Facebook using your name but with a different photo, different date of birth, and different hometown. Create enough doppelgangers to add noise to the search results for your name. And have them share embarrassing stories on their blogs. The goal is to ensure that the din of your alternates drowns out anything embarrassing showing up for you.

Although it will look suspicious if you're the only name on Google with such chaff. So clearly you must also do this for your friends and family. Really you'll be doing them a favor.

HTML5 Table Flipper Experiment

2012-03-02T00:00:00Z

The goal of this experiment was to combine the flipping tables emoticons with the Threw It On The Ground video using shiny new HTML5-ish features and the end result is the table flipper flipping the Threw It On the Ground video.

The table flipper emoticon is CSS before content that changes on hover. Additionally on hover a CSS transform is applied to flip the video upside down several times and move it to the right and there's a CSS transition to animate the flipping. The only issue I ran into is that (at least on Windows) Flash doesn't like to have CSS transform rotations applied to it. So to get the most out of the flip experiment you must opt-in to HTML5 video on YouTube. And of course you must use a browser that supports the various things I just mentioned, like the latest Chrome (or not yet released IE10).

Glitch Helperator

2012-02-29T00:00:00Z

I've been working on the Glitch Helperator. It is a collection of tools and things I've put together for Glitch. It has a few features that I haven't seen elsewhere including:

Favorite Streets: A notebook in which you can save information about interesting streets and later use it to find your way back to them.
Birthday: Find out how old your Glitch is and the date of your next birthday in Glitch time or Earth time.
API Update History: A history of changes to the streets, skills and achievements of Glitch noting when new ones are added and when existing ones are changed.

It also has an interactive skill tree, find nearest feature tool, and achievement display. If you play Glitch, check it out.

Client Side Cross Domain Data YQL Hack

2012-02-27T00:00:00Z

One of the more limiting issues of writing client side script in the browser is the same origin limitations of XMLHttpRequest. The latest version of all browsers support a subset of CORS to allow servers to opt-in particular resources for cross-domain access. Since IE8 there's XDomainRequest and in all other browsers (including IE10) there's XHR L2's cross-origin request features. But the vast majority of resources out on the web do not opt-in using CORS headers and so client side only web apps like a podcast player or a feed reader aren't doable.

One hack-y way around this I've found is to use YQL as a CORS proxy. YQL applies the CORS header to all its responses and among its features it allows a caller to request an arbitrary XML, HTML, or JSON resource. So my network helper script first attempts to access a URI directly using XDomainRequest if that exists and XMLHttpRequest otherwise. If that fails it then tries to use XDR or XHR to access the URI via YQL. I wrap my URIs in the following manner, where type is either "html", "xml", or "json":

        yqlRequest = function(uri, method, type, onComplete, onError) {
            var yqlUri = "http://query.yahooapis.com/v1/public/yql?q=" + 
                encodeURIComponent("SELECT * FROM " + type + ' where url="' + encodeURIComponent(uri) + '"');

            if (type == "html") {
                yqlUri += encodeURIComponent(" and xpath='/*'");
            }
            else if (type == "json") {
                yqlUri += "&callback=&format=json";
            }
            ...

This also means I can get JSON data itself without having to go through JSONP.

Web Worker Initialization Race

2012-02-24T00:00:00Z

Elaborating on a previous brief post on the topic of Web Worker initialization race conditions, there's two important points to avoid a race condition when setting up a Worker:

The parent starts the communication posting to the worker.
The worker sets up its message handler in its first synchronous block of execution.

For example the following has no race becaues the spec guarentees that messages posted to a worker during its first synchronous block of execution will be queued and handled after that block. So the worker gets a chance to setup its onmessage handler. No race:

'parent.js':
   var worker = new Worker();
   worker.postMessage("initialize");

'worker.js':
   onmessage = function(e) {
      // ...
   }

The following has a race because there's no guarentee that the parent's onmessage handler is setup before the worker executes postMessage. Race (violates 1):

'parent.js':
   var worker = new Worker();
   worker.onmessage = function(e) {
      // ...
   };

'worker.js':
   postMessage("initialize");

The following has a race because the worker has no onmessage handler set in its first synchronous execution block and so the parent's postMessage may be sent before the worker sets its onmessage handler. Race (violates 2):

'parent.js':
   var worker = new Worker();
   worker.postMessage("initialize");

'worker.js':
   setTimeout(
      function() {
         onmessage = function(e) {
            // ...
         }
      },
      0);

Prime HTTP Status Codes

2012-02-22T00:00:00Z

These are the prime HTTP status codes:

URI Percent Encoding Ignorance Level 2 - There is no Unencoded URI

2012-02-20T00:00:00Z

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Getting into the more subtle levels of URI percent-encoding ignorance, folks try to apply their knowledge of percent-encoding to URIs as a whole producing the concepts escaped URIs and unescaped URIs. However there are no such things - URIs themselves aren't percent-encoded or decoded but rather contain characters that are percent-encoded or decoded. Applying percent-encoding or decoding to a URI as a whole produces a new and non-equivalent URI.

Instead of lingering on the incorrect concepts we'll just cover the correct ones: there's raw unencoded data, non-normal form URIs and normal form URIs. For example:

http://example.com/%74%68%65%3F%70%61%74%68?query
http://example.com/the%3Fpath?query
"http", "example.com", "the?path", "query"

In the above (A) is not an 'encoded URI' but rather a non-normal form URI. The characters of 'the' and 'path' are percent-encoded but as unreserved characters specific in the RFC should not be encoded. In the normal form of the URI (B) the characters are decoded. But (B) is not a 'decoded URI' -- it still has an encoded '?' in it because that's a reserved character which by the RFC holds different meaning when appearing decoded versus encoded. Specifically in this case, it appears encoded which means it is data -- a literal '?' that appears as part of the path segment. This is as opposed to the decoded '?' that appears in the URI which is not part of the path but rather the delimiter to the query.

Usually when developers talk about decoding the URI what they really want is the raw data from the URI. The raw decoded data is (C) above. The only thing to note beyond what's covered already is that to obtain the decoded data one must parse the URI before percent decoding all percent-encoded octets.

Of course the exception here is when a URI is the raw data. In this case you must percent-encode the URI to have it appear in another URI. More on percent-encoding while constructing URIs later.

Why I Like Glitch

2012-02-17T00:00:00Z

Sarah and I have been enjoying Glitch for a while now. Reviews are usually positive although occasionally biting (but mostly accurate).

I enjoy Glitch as a game of exploration: exploring the game's lands with hidden and secret rooms, and exploring the games skills and game mechanics. The issue with my enjoyment coming from exploration is that after I've explored all streets and learned all skills I've got nothing left to do. But I've found that even after that I can have fun writing client side JavaScript against Glitch's web APIs making tools (I work on the Glitch Helperator) for use in Glitch. And on a semi-regular basis they add new features reviving my interest in the game itself.

URI Percent-Encoding Ignorance Level 1 - Purpose

2012-02-15T00:00:00Z

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping).

Worse than the lame blog comments hating on percent-encoding is the shipping code which can do actual damage. In one very large project I won't name, I've fixed code that decodes all percent-encoded octets in a URI in order to get rid of pesky percents before calling ShellExecute. An unnamed developer with similar intent but clearly much craftier did the same thing in a loop until the string's length stopped changing. As it turns out percent-encoding serves a purpose and can't just be removed arbitrarily.

Percent-encoding exists so that one can represent data in a URI that would otherwise not be allowed or would be interpretted as a delimiter instead of data. For example, the space character (U+0020) is not allowed in a URI and so must be percent-encoded in order to appear in a URI:

http://example.com/the%20path/
http://example.com/the path/

In the above the first is a valid URI while the second is not valid since a space appears directly in the URI. Depending on the context and the code through which the wannabe URI is run one may get unexpected failure.

For an additional example, the question mark delimits the path from the query. If one wanted the question mark to appear as part of the path rather than delimit the path from the query, it must be percent-encoded:

http://example.com/foo%3Fbar
http://example.com/foo?bar

In the second, the question mark appears plainly and so delimits the path "/foo" from the query "bar". And in the first, the querstion mark is percent-encoded and so the path is "/foo%3Fbar".

Blackmail DRM - Stolen Thoughts

2012-02-13T00:00:00Z

Most existing DRM attempts to only allow the user to access the DRM'ed content with particular applications or with particular credentials so that if the file is shared it won't be useful to others. A better solution is to encode any of the user's horrible secrets into unique versions of the DRM'ed content so that the user won't want to share it. Entangle the users and the content provider's secrets together in one document and accordingly their interests. I call this Blackmail DRM. For an implementation it is important to point out that the user's horrible secret doesn't need to be verified as accurate, but merely verified as believable.

Apparently I need to get these blog posts written faster because only recently I read about Social DRM which is a light weight version of my idea but with a misleading name. Instead of horrible secrets, they say they'll use personal information like the user's name in the DRM'ed content. More of my thoughts stolen and before I even had a chance to think of it first!

URI Percent Encoding Ignorance Level 0 - Existence

2012-02-10T00:00:00Z

As a professional URI aficionado I deal with various levels of ignorance on URI percent-encoding (aka URI encoding, or URL escaping). The basest ignorance is with respect to the mere existence of percent-encoding. Percents in URIs are special: they always represent the start of a percent-encoded octet. That is to say, a percent is always followed by two hex digits that represents a value between 0 and 255 and doesn't show up in a URI otherwise.

The IPv6 textual syntax for scoped addresses uses the '%' to delimit the zone ID from the rest of the address. When it came time to define how to represent scoped IPv6 addresses in URIs there were two camps: Folks who wanted to use the IPv6 format as is in the URI, and those who wanted to encode or replace the '%' with a different character. The resulting thread was more lively than what shows up on the IETF URI discussion mailing list. Ultimately we went with a percent-encoded '%' which means the percent maintains its special status and singular purpose.

JavaScript Array methods in the latest browsers

2011-12-03T00:00:00Z

Cool and (relatively) new methods on the JavaScript Array object are here in the most recent versions of your favorite browser! More about them on ECMAScript5, MSDN, the IE blog, or Mozilla's documentation. Here's the list that's got me excited:

some & every: Does your callback function return true for any (some) or all (every) of the array's elements?
filter: Filters out elements for which your callback function returns false (in a new copy of the Array).
map: Each element is replaced with the result of it run through your callback function (in a new copy of the Array).
reduce & reduceRight: Your callback is called on each element in the array in sequence (from start to finish in reduce and from finish to start in reduceRight) with the result of the previous callback call passed to the next. Reduce your array to a single value aggregated in any manner you like via your callback function.
forEach: Simply calls your callback passing in each element of your array in turn. I have vague performance concerns as compared to using a normal for loop.
indexOf & lastIndexOf: Finds the first or last (respectively) element in the array that matches the provided value via strict equality operator and returns the index of that element or -1 if there is no such element. Surprisingly, no custom comparison callback method mechanism is provided.

Bug Spotting: Ctors with default parameters

2011-12-01T00:00:00Z

The following code compiled just fine but did not at all act in the manner I expected:

BOOL CheckForThing(__in CObj *pObj, __in IFigMgr* pFigMgr, __in_opt LPCWSTR url)
{
    BOOL fCheck = FALSE;
    if (SubCheck(pObj))
    {
        ...

I’m calling SubCheck which looks like:

bool SubCheck(const CObj& obj);

Did you spot the bug? As you can see I should be passing in *pObj not pObj since the method takes a const CObj& not a CObj*. But then why does it compile?

It works because CObj has a constructor with all but one param with default values and CObj is derived from IUnknown:

CObj(__in_opt IUnknown * pUnkOuter, __in_opt LPCWSTR pszUrl = NULL);

Accordingly C++ uses this constructor as an implicit conversion operator. So instead of passing in my CObj, I end up creating a new CObj on the stack passing in the CObj I wanted as the outer object which has a number of issues.

The lesson is unless you really want this behavior, don't make constructors with all but 1 or 0 default parameters. If you need to do that consider using the 'explicit' keyword on the constructor.

More info about forcing single argument constructors to be explicit is available on stack overflow.

Replacing Google Reader Shared Feeds with Tumblr

2011-11-28T00:00:00Z

Last time I wrote about how I switched from Delicious to Google Reader's shared links feature only to find out that week that Google was removing the Google Reader shared links feature in favor of Google Plus social features (I'll save my Google Plus rant for another day).

Forced to find something new again, I'm now very pleased with Tumblr. Google Reader has Tumblr in its preset list of Send To sites which makes it relatively easy to add articles. And Tumblr's UX for adding things lets me easily pick a photo or video to display from the article - something which I had put together with a less convenient UX on my bespoke blogging system. For adding things outside of Google Reader I made a Tumblr accelerator to hookup to the Tumblr Add UX.

Of course they have an RSS feed which I hooked up to my blog. The only issue I had there is that when you add a link (and not a video or photo) to Tumblr, the RSS feed entry title for that link is repeated in the entry description as a link followed by a colon and then the actual description entered into Tumblr. I want my title separate so I can apply my own markup so I did a bit of parsing of the description to remove the repeated title from the description.

URI Empty Path Segments Matter

2011-11-23T00:00:00Z

Shortly after joining the Internet Explorer team I got a bug from a PM on a popular Microsoft web server product that I'll leave unnamed (from now on UWS). The bug said that IE was handling empty path segments incorrectly by not removing them before resolving dotted path segments. For example UWS would do the following:

A.1. http://example.com/a/b//../
A.2. http://example.com/a/b/../
A.3. http://example.com/a/

In step 1 they are given a URI with dotted path segment and an empty path segment. In step 2 they remove the empty path segment, and in step 3 they resolve the dotted path segment. Whereas, given the same initial URI, IE would do the following:

B.1. http://example.com/a/b//../
B.2. http://example.com/a/b/

IE simply resolves the dotted path segment against the empty path segment and removes them both. So, how did I resolve this bug? As "By Design" of course!

The URI RFC allows path segments of zero length and does not assign them any special meaning. So generic user agents that intend to work on the web must not treat an empty path segment any different from a path segment with some text in it. In the case above IE is doing the correct thing.

That's the case for generic user agents, however servers may decide that a URI with an empty path segment returns the same resource as a the same URI without that empty path segment. Essentially they can decide to ignore empty path segments. Both IIS and Apache work this way and thus return the same resource for the following URIs:

http://exmaple.com/foo//bar///baz
http://example.com/foo/bar/baz

The issue for UWS is that it removes empty path segments before resolving dotted path segments. It must follow normal URI procedure before applying its own additional rules for empty path segments. Not doing that means they end up violating URI equivalency rules: URIs (A.1) and (B.2) are equivalent but UWS will not return the same resource for them.

Features of image type input tags in HTML

2011-11-21T00:00:00Z

A bug came up the other day involving markup containing <input type="image" src="http://example.com/.... I knew that "image" was a valid input type but it wasn't until that moment that I realized I didn't know what it did. Looking it up I found that it displays the specified image and when the user clicks on the image, the form is submitted with an additional two name value pairs: the x and y positions of the point at which the user clicked the image.

Take for example the following HTML:

<form action="http://example.com/">
<input type="image" name="foo" src="http://deletethis.net/dave/images/davebefore.jpg">
</form>

If the user clicks on the image, the browser will submit the form with a URI like the following:http://example.com/?foo.x=145&foo.y=124.

This seemed like an incredibly specific feature to be built directly into the language when this could instead be done with javascript. I looked a bit further and saw that its been in HTML since at least HTML2, which of course makes much more sense. Javascript barely existed at that point and sending off the user's click location in a form may have been the only way to do something interesting with that action.

Replacing Delicious with Google Reader

2011-11-17T00:00:00Z

I had previously replaced my use of Delicious with Google Reader. Delicious had a number of issues during their switch over from Yahoo to the new owners and I was eventually fed up enough to remove it from daily use. I used Delicious to do the following things:

Create a list of things to read later
Save things to read again in the future
Search through things I read and enjoyed (esp via tags)
Annotate and share things on my blog

I realized that since I did most of my web browsing in Google Reader now anyway I may as well make use of its features. I star things to note I want to read it later or save to read again later. I can annotate with notes in Google Reader and I can share items to my web site by way of the shared items feed. Additionally for when I'm not in Google Reader there's a bookmarklet to add an arbitrary web site as a shared item in Google Reader.

Of course I wrote this and switched over about 1 week before Google removed the sharing feature from Google Reader. I'm irritated but in practice it forced me to find a different option which has worked out mostly better. New blog post coming soon about that...

Haven't Been Posting Much

2011-10-18T00:00:00Z

I haven't been updating my blog recently. But I have three excellent reasons:

The manner in which I use Delicious is broken.
I've been playing Glitch instead of reading through Google Reader.
I have a newborn!

WPAD Server Fiddler Extension Update v1.0.1

2011-06-12T00:00:00Z

As it turns out the WPAD Server Fiddler Extension I made a while back actually has a non-malicious purpose. Apparently its useful for debugging HTTP on the WP7 phone (or so I'm told). Anyway I took some requests and I've fixed a few minor bugs (start button not updating correctly), changed the dialog to be a Fiddler tab so you can use it non-modally, and the WPAD server is now always off when Fiddler starts.

Command line for finding missing URLACTIONs

2011-05-28T00:00:00Z

I wanted to ensure that my switch statement in my implementation of IInternetSecurityManager::ProcessURLAction had a case for every possible documented URLACTION. I wrote the following short command line sequence to see the list of all URLACTIONs in the SDK header file not found in my source file:

grep URLACTION urlmon.idl | sed 's/.*\(URLACTION[a-zA-Z0-9_]*\).*/\1/g;' | sort | uniq > allURLACTIONs.txt
grep URLACTION MySecurityManager.cpp | sed 's/.*\(URLACTION[a-zA-Z0-9_]*\).*/\1/g;' | sort | uniq > myURLACTIONs.txt
comm -23 allURLACTIONs.txt myURLACTIONs.txt

I'm not a sed expert so I had to read the sed documentation, and I heard about comm from Kris Kowal's blog which happilly was in the Win32 GNU tools pack I already run.

But in my effort to learn and use PowerShell I found the following similar command line:

diff 
(more urlmon.idl | %{ if ($_ -cmatch "URLACTION[a-zA-Z0-9_]*") { $matches[0] } } | sort -uniq) 
(more MySecurityManager.cpp | %{ if ($_ -cmatch "URLACTION[a-zA-Z0-9_]*") { $matches[0] } } | sort -uniq)

In the PowerShell version I can skip the temporary files which is nice. 'diff' is mapped to 'compare-object' which seems similar to comm but with no parameters to filter out the different streams (although this could be done more verbosely with the ?{ } filter syntax). In PowerShell uniq functionality is built into sort. The builtin -cmatch operator (c is for case sensitive) to do regexp is nice plus the side effect of generating the $matches variable with the regexp results.

clip.exe - Useful tool I didn't know shipped with Windows

2011-05-26T00:00:00Z

When you run clip.exe, whatever comes into its standard input is put onto the clipboard. So when you need to move the result of something in your command window somewhere else you can pipe the result into clip.exe. Then you won't have to worry about the irritating way cmd.exe does block copy/pasting and you avoid having to manually fixup line breaks in wrapped lines. For instance, you can put the contents of a script into the clipboard with:

more cdo.cmd | clip

I've got a lot of stuff dumped in my bin folder that I sync across all my PCs so I didn't realize that clip.exe is a part of standard Windows installs.

Nice for avoiding the block copy in cmd.exe but I'd prefer to have the contents sort of tee'd into the clipboard and standard output. So TeeClip.ps1:

$input | tee -var teeclipout | clip;
$teeclipout;

_opt Mnemonic

2011-05-24T00:00:00Z

I always have trouble remembering where the opt goes in SAL in the __deref_out case. The mnemonic is pretty simple: the _opt at the start of the SAL is for the pointer value at the start of the function. And the _opt at the end of the SAL is for the dereferenced pointer value at the end of the function.

SAL	foo == nullptr allowed at function start?	*foo == nullptr allowed at function end?
__deref_out void **foo	No	No
__deref_opt_out void **foo	Yes	No
__deref_out_opt void **foo	No	Yes
__deref_opt_out_opt void **foo	Yes	Yes

PowerShell Script Batch File Wrapper

2011-05-22T00:00:00Z

I'm trying to learn and use PowerShell more, but plenty of other folks I know don't use PowerShell. To allow them to use my scripts I use the following cmd.exe batch file to make it easy to call PowerShell scripts. To use, just name the batch file name the same as the corresponding PowerShell script filename and put it in the same directory.

@echo off
if "%1"=="/?" goto help
if "%1"=="/h" goto help
if "%1"=="-?" goto help
if "%1"=="-h" goto help

%systemroot%\system32\windowspowershell\v1.0\powershell.exe -ExecutionPolicy RemoteSigned -Command . %~dpn0.ps1 %*
goto end

:help
%systemroot%\system32\windowspowershell\v1.0\powershell.exe -ExecutionPolicy RemoteSigned -Command help %~dpn0.ps1 -full
goto end

:end

Additionally for PowerShell scripts that modify the current working directory I use the following batch file:

@echo off
if "%1"=="/?" goto help
if "%1"=="/h" goto help
if "%1"=="-?" goto help
if "%1"=="-h" goto help

%systemroot%\system32\windowspowershell\v1.0\powershell.exe -ExecutionPolicy RemoteSigned -Command . %~dpn0.ps1 %*;(pwd).Path 1> %temp%\%~n0.tmp 2> nul
set /p newdir=

Capturing HTTPS with FiddlerCore

2011-04-06T00:00:00Z

I used FiddlerCore in GeolocMock to edit HTTPS responses and ran into two stumbling blocks that I'll document here. The first is that I didn't check if the Fiddler root cert existed or was installed, which of course is necessary to edit HTTPS traffic. The following is my code where I check for the certs.

    if (!Fiddler.CertMaker.rootCertExists())
    {
        if (!Fiddler.CertMaker.createRootCert())
        {
            throw new Exception("Unable to create cert for FiddlerCore.");
        }
    }

    if (!Fiddler.CertMaker.rootCertIsTrusted())
    {
        if (!Fiddler.CertMaker.trustRootCert())
        {
            throw new Exception("Unable to install FiddlerCore's cert.");
        }
    }

The second problem I had (which would have been solved had I read all the sample code first) was that my changes weren't being applied. In my app I only need the BeforeResponse but in order to modify the response I must also sign up for the BeforeRequest event and mark the bBufferResponse flag on the session before the response comes back. For example:

    Fiddler.FiddlerApplication.BeforeRequest += new SessionStateHandler(FiddlerApplication_BeforeRequest);
    Fiddler.FiddlerApplication.BeforeResponse += new SessionStateHandler(FiddlerApplication_BeforeResponse);
...
    private void FiddlerApplication_BeforeRequest(Session oSession)
    {
        if (IsInterestingSession(oSession))
        {
            oSession.bBufferResponse = true;
        }
    }

JavaScript & .NET interop via WebBrowser Control

2011-04-05T00:00:00Z

For my GeolocMock weekend project I intended to use the Bing Maps API to display a map in a WebBrowser control and allow the user to interact with that to select a location to be consumed by my application. Getting my .NET code to talk to the JavaScript in the WebBrowser control was surprisingly easy.

To have .NET execute JavaScript code you can use the InvokeScript method passing the name of the JavaScript function to execute and an object array of parameters to pass:

this.webBrowser2.Document.InvokeScript("onLocationStateChanged",
   new object[] {
      latitudeTextBoxText,
      longitudeTextBoxText,
      altitudeTextBoxText,
      uncertaintyTextBoxText
   });

The other direction, having JavaScript call into .NET is slightly more complicated but still pretty easy as far as language interop goes. The first step is to mark your assembly as ComVisible so that it can interact with JavaScript via COM. VS had already added a ComVisible declaration to my project I just had to change the value to true.

[assembly: ComVisible(true)]

Next set ObjectForScripting attribute to the object you want to expose to JavaScript.

this.webBrowser2.ObjectForScripting = this.locationState;

Now that object is exposed as window.external in JavaScript and you can call methods on it.

window.external.Set(lat, long, alt, gUncert);

However you don't seem to be able to test for the existence of methods off of it. For example the following JavaScript generates an exception for me even though I have a Set method:

if (window.external && window.external.Set) {

IE9 Document Mode in WebOC

2011-04-04T00:00:00Z

Working on GeolocMock it took me a bit to realize why my HTML could use the W3C Geolocation API in IE9 but not in my WebBrowser control in my .NET application. Eventually I realized that I was getting the wrong IE doc mode. Reading this old More IE8 Extensibility Improvements IE blog post from the IE blog I found the issue is that for app compat the WebOC picks older doc modes but an app hosting the WebOC can set a regkey to get different doc modes. The IE9 mode isn't listed in that article but I took a guess based on the values there and the decimal value 9999 gets my app IE9 mode. The following is the code I run in my application to set its regkey so that my app can get the IE9 doc mode and use the geolocation API.

        static private void UseIE9DocMode()
        {
            RegistryKey key = null;
            try
            {
                key = Registry.CurrentUser.OpenSubKey("Software\\Microsoft\\Internet Explorer\\Main\\FeatureControl\\FEATURE_BROWSER_EMULATION", true);
            }
            catch (Exception)
            {
                key = Registry.CurrentUser.CreateSubKey("Software\\Microsoft\\Internet Explorer\\Main\\FeatureControl\\FEATURE_BROWSER_EMULATION");
            }
            key.SetValue(System.Diagnostics.Process.GetCurrentProcess().MainModule.ModuleName, 9999, RegistryValueKind.DWord);
            key.Close();
        }

GeolocMock Tool - Tell IE9 Where You Are

2011-04-03T00:00:00Z

I've made GeolocMock. If your PC has no geolocation devices, IE9 uses a webservice to determine your location. GeolocMock uses FiddlerCore to intercept the response from the webservice and allows the user to replace the location in the response with another. This was a fun weekend project in order to play with FiddlerCore, the W3C Geoloc APIs in IE9, hosting the IE9 WebOC in a .NET app, and the Bing Maps APIs.

Listen to Max Tannone and Watch Moon

2011-02-23T00:00:00Z

Max Tannone does awesome remix albums:

Jaydiohead - Radiohead & Jay-Z mashup. Listen:
Jaydiohead - 06 - Dirt Off Your Android by Max Tannone

Selene - Richard Rich & beats over the Moon soundtrack. Listen:
Selene - 01 - You Are Here by Max Tannone

Double Check Your Head - Beastie Boys albums mixed together. Listen:
Beastie Boys - 02 - Pass The Gratitude by Max Tannone

Also, the movie Moon is really good on a variety of points. Sam Rockwell and the voice of Kevin Spacey! Its available on Netflix Watch Instantly so you have no excuse!

Ratatat Rocks

2010-12-28T00:00:00Z

I just found out that I like the group Ratatat. I'd first heard them way back when the Zune was first released as the backing for Los Corazones on the zune-arts.net website.

But I didn't know who they were until today when I watched this Filmography 2010 video (via Kottke)

Until about 1:16 in, the music is Ratatat's Nostrand. On the first viewing it drove me crazy because I could only vaguely recall hearing something like that music before. I tracked it down via the zune-arts thing above and eventually found my way to the Nostrand video. Funny, all the recent comments on that one are from people who also just watched the Filmography video.

Console Build Window Jump Lists Tool

2010-12-13T00:00:00Z

I've made two simple command line tools related to the console window and Win7 jump lists. The source is available for both but neither is much more than the sort of samples you'd find on MSDN =).

SetAppUserModelId lets you change the Application User Model ID for the current console window. The AppUserModelId is the value Win7 uses to group together icons on the task bar and is what the task bar's jump lists are associated with. The tool lets you change that as well as the icon and name that appear in the task bar for the window, and the command to launch if the user attempts to re-launch the application from its task bar icon.

SetJumpList lets you set the jump list associated with a particular AppUserModelId. You pass the AppUserModelId as the only parameter and then in its standard input you give it lines specifying items that should appear in the jump list and what to execute when those items are picked.

I put these together to make my build environment easier to deal with at work. I have to deal with multiple enlistments in many different branches and so I wrote a simple script around these two tools to group my build windows by branch name in the task bar, and to add the history of commands I've used to launch the build environment console windows to the jump list of each.

Windows 7 Accelerator Platform COM / C# Interop

2010-08-20T00:00:00Z

For a new project I'm working on involving IE's installed Accelerators and OpenSearch search providers via the Windows 7 Accelerator Platform, I've created a C#/COM interop class for those APIs.

Download the osinterop.cs interop file here.

DVD Ripping and Viewing in Windows Media Center

2010-08-17T00:00:00Z

I've just got a new media center PC connected directly to my television with lots of HD space and so I'm ripping a bunch of my DVDs to the PC so I don't have to fuss with the physical media. I'm ripping with DVD Rip, viewing the results in Windows 7's Windows Media Center after turning on the WMC DVD Library, and using a powershell script I wrote to copy over cover art and metadata.

My powershell script follows. To use it you must do the following:

Run Windows Media Center with the DVD in the drive and view the disc's metadata info.
Rip each DVD to its own subdirectory of a common directory.
The name of the subdirectory to which the DVD is ripped must have the same name as the DVD name in the metadata. An exception to this are characters that aren't allowed in Windows paths (e.g. <, >, ?, *, etc)
Run the script and pass the path to the common directory containing the DVD rips as the first parameter.

Running WMC and viewing the DVD's metadata forces WMC to copy the metadata off the Internet and cache it locally. After playing with Fiddler and reading this blog post on WMC metadata I made the following script that copies metadata and cover art from the WMC cache to the corresponding DVD rip directory.

Download copydvdinfo.ps1

Google and Outlook Calendar Sync'ing

2010-07-08T00:00:00Z

I previously described my desire to hook my Outlook calendar up to my Google calendar. I just found out that I can do this and the reverse as both support publishing calendars to the Internet. The following are how I set this up under Outlook 2010 and Google Calendar:

In Outlook, I go to the calendar view, right click on my calendar and select "Share Publish to Office.com". At this point I can change the permissions to allow anonymous Internet access, and under Detail change between 'Full details' (full calendar), 'Limited details' (subject lines & availability only), 'Availability only'. Availability only is almost just what I want -- I'd also like to include location but availability only is good enough. After hitting OK here I get a 'Do you want to send an invitation...' dialog box. I hit 'Yes' and I can copy the webcals:// URL out of the email window that opens up. Next, to add it to my Google calendar, I open http://www.google.com/calendar/, and under 'Other calendars', I select 'Add Add by URL', paste in that webcals:// URL but change the 'webcals' at the start to 'https'.

In Google Calendar, I can click on my calendar name under 'My calendars', select 'Calendar settings', and on the new page, look under 'Calendar Address', click the ICAL icon, and copy the URL in the new dialog. Now back in Outlook I go to the Calendar view, right click on 'My Calendars', and select 'Add Calendar From Internet...'. In the new dialog that pops up I paste in the URL from Google Calendar.

In this fashion I can share public calendar data between my personal and work calendars.

deletethis.net

Edge and WebView2 in Chrome DevTools MCP

Edge Launch #

Edge Auto-connect #

WebView2 Auto-connect #

WebView2 wv2winrt bug case study

Code Search Comparison: GitHub, Azure DevOps, and Chromium

Comparison Table #

Example Comparison #

Notable Differences: #

kTopLevel #

NOT path:*test* #

NOT ext:mojom #

-path:^out/ #

WatchBuildLog VS Code Extension

Development #

Time Travel Debugging WPT on Windows

OpenSourceIn Browser Extension

Development #

Side project fatigue mitigation #

Creating C grade skeleton #

Works best when you know what you want #

Noobs WPT in Chromium learnings

Background #

WPT learnings #

Wildcards #

Sync'ing between chromium and WPT #

Manifest update 30s #

Virtual test folders #

WPT name suffix .https.html #

Testdriver browser extension test_driver #

Debugging #

content_shell #

New test stability #

Command line summary #

Eleventy on GitHub Pages (Part 3)

CSS Scroll Animated Sticky Header

Eleventy on GitHub Pages (Part 2)

Eleventy on GitHub Pages

Restricted Capabilities and MSIX APIs

404 Good News, Bad News

Watch JavaScript via Proxy

How this works #

What you can do with it #

Watch a for loop #

Watch instanceof #

JavaScript Arrays are neat

Right-To-Left Override Twitter Name

Scrollbars in EdgeHtml WebView and Edge browser

Edge browser and JavaScript UWP app security model comparison

Process Model

ApplicationContentUriRules

GoBack/GoForward in Win10 UWP WebView

Changing the User Agent string in UWP WebView

Windows.Web.UI.Interop.WebViewControl localhost access

Win10 PWA Terminology

Web App

Windows Web App

Packaged vs Hosted Web App

Win10 PWA

Tiny browser features: JSBrowser crash resistance

Tiny browser features: JSBrowser zoom

Slider

Scale

Multiple Windows in Win10 JavaScript UWP apps

Win10 Changes

WinRT viewId

Delaying Visibility

Primary Window Differences

Same Origin Communication Restrictions

MSApp.getHtmlPrintDocumentSourceAsync - JavaScript UWP app printing

Background

Sample

Win10 UWP WebView AddWebAllowedObject details

Lifetime

WinRT access

Object requirements

Property name

Errors

Application Content URI Rule effects

`kTopLevel` #

`NOT path:test` #

`NOT ext:mojom` #

`-path:^out/` #

WPT name suffix `.https.html` #

Testdriver browser extension `test_driver` #

Watch a `for` loop #

Watch `instanceof` #