over - Dave's Blog

Its rare to find devs anticipating Unicode control characters showing up in user input. And the most fun when unanticipated is the Right-To-Left Override character U+202E. Unicode characters have an implicit direction so that for example by default Hebrew characters are rendered from right to left, and English characters are rendered left to right. The override characters force an explicit direction for all the text that follows.

I chose my Twitter display name to include the HTML encoding of the Right-To-Left Override character #x202E; as a sort of joke or shout out to my favorite Unicode control character. I did not anticipate that some Twitter clients in some of their UI would fail to encode it correctly. There's no way I can remove that from my display name now.

Try it on Amazon.

The scrollbars in UWP WebView and in Edge have different default behavior leading to many emails to my team. (Everything I talk about here is for the EdgeHtml based WebView and Edge browser and does not apply to the Chromium based Edge browser and WebView2).

There is a Edge only -ms-overflow-style CSS property that controls scroll behavior. We have a different default for this in the WebView as compared to the Edge browser. If you want the appearance of the scrollbar in the WebView to match the browser then you must explicitly set that CSS property. The Edge browser default is scrollbar which gives us a Windows desktop styled non-auto-hiding scrollbar. The WebView default is -ms-autohiding-scrollbar which gives a sort of compromise between desktop and UWP app scrollbar behavior. In this configuration it is auto-hiding. When used with the mouse you'll get Windows desktop styled scrollbars and when used with touch you'll get the UWP styled scrollbars.

Since WebViews are intended to be used in apps this style is the default in order to better match the app's scrollbars. However this difference between the browser and WebView has led to confusion.

Here’s an -ms-overflow-style JSFiddle showing the difference between the two styles. Try it in the Edge browser and in WebView. An easy way to try it in the Edge WebView is using the JavaScript Browser.

There's no perfect way to change the user agent string for the UWP WebView (x-ms-webview in HTML, Windows.UI.Xaml.Controls.WebView in XAML, and Windows.Web.UI.Interop.WebViewControl in Win32) but there are two imperfect methods folks end up using.

The first is to call UrlMkSetSessionOption. This is an old public API that allows you to configure various arcane options including one that is the default user agent string for requests running through urlmon. This API is allowed by the Microsoft Store for UWP apps. The change it applies is process wide which has two potential drawbacks. If you want to be able to have different UA strings set for different requests from a WebView that's not really possible with this solution. The other drawback is if you're using out of process WebView, you need to ensure you're calling into UrlMkSetSessionOption in the WebView's process. You'll need to write third party WinRT that calls UrlMkSetSessionOption, create the out of proc WebView, navigate it to some trusted local page, use AddWebAllowedObject or provide that URI WinRT access, and call into your third party WinRT. You'll need to do that for any new WebView process you create.

The second less generally applicable solution is to use NavigateWithHttpRequestMessage and set the User-Agent HTTP header. In this case you get to control the scope of the user agent string changes but has the limitations that not all sub resource downloads will use this user agent string and for navigations you don't initiate you have to manually intercept and re-request being careful to transfer over all POST body state and HTTP headers correctly. That last part is not actually possible for iframes.

Folks familiar with JavaScript UWP apps in Win10 have often been confused by what PWAs in Win10 actually are. TLDR: PWAs in Win10 are simply JavaScript UWP apps. The main difference between these JS UWP Apps and our non-PWA JS UWP apps are our target end developer audience, and how we get Win10 PWAs into the Microsoft Store. See this Win10 blog post on PWAs on Win10 for related info.

Web App

On the web a subset of web sites are web apps. These are web sites that have app like behavior - that is a user might call it an app like Outlook, Maps or Gmail. And they may also have a W3C app manifest.

A subset of web apps are progressive web apps. Progressive web apps are web apps that have a W3C app manifest and a service worker. Various OSes are beginning to support PWAs as first class apps on their platform. This is true for Win10 as well in which PWAs are run as a WWA.

Windows Web App

In Win10 a WWA (Windows Web App) is an unofficial term for a JavaScript UWP app. These are UWP apps so they have an AppxManifest.xml, they are packaged in an Appx package, they run in an App Container, they use WinRT APIs, and are installed via the Microsoft Store. Specific to WWAs though, is that the AppxManifest.xml specifies a StartPage attribute identifying some HTML content to be used as the app. When the app is activated the OS will create a WWAHost.exe process that hosts the HTML content using the EdgeHtml rendering engine.

Packaged vs Hosted Web App

Within that we have a notion of a packaged web app and an HWA (hosted web app). There's no real technical distinction for the end developer between these two. The only real difference is whether the StartPage identifies remote HTML content on the web (HWA), or packaged HTML content from the app's appx package (packaged web app). An end developer may create an app that is a mix of these as well, with HTML content in the package and HTML content from the web. These terms are more like ends on a continuum and identifying two different developer scenarios since the underlying technical aspect is pretty much identical.

Win10 PWA

Win10 PWAs are simply HWAs that specify a StartPage of a URI for a PWA on the web. These are still JavaScript UWP apps with all the same behavior and abilities as other UWP apps. We have two ways of getting PWAs into the Microsoft Store as Win10 PWAs. The first is PWA Builder which is a tool that helps PWA end developers create and submit to the Microsoft Store a Win10 PWA appx package. The second is a crawler that runs over the web looking for PWAs which we convert and submit to the Store using an automated PWA Builder-like tool to create a Win10 PWA from PWAs on the web (see Welcoming PWAs to Win10 for more info). In both cases the conversion involves examining the PWAs W3C app manifest and producing a corresponding AppxManifest.xml. Not all features supported by AppxManifest.xml are also available in the W3c app manifest. But the result of PWA Builder can be a working starting point for end developers who can then update the AppxManifest.xml as they like to support features like share targets or others not available in W3C app manifests.

JSBrowser is a basic browser built as a Win10 JavaScript UWP app around the WebView HTML element. Its fun and relatively simple to implement tiny browser features in JavaScript and in this post I'm implementing crash resistance.

The normal DOM mechanisms for creating an HTML WebView create an in-process WebView, in which the WebView runs on a unique UI thread. But we can use the MSWebView constructor instead to create an out-of-process WebView in which the WebView runs in its own distinct WebView process. Unlike an in-process WebView, Web content running in an out-of-process WebView can only crash the WebView process and not the app process.

        this.replaceWebView = () => {
            let webview = document.querySelector("#WebView");
            // Cannot access webview.src - anything that would need to communicate with the webview process may fail
            let oldSrc = browser.currentUrl;
            const webviewParent = webview.parentElement;
            webviewParent.removeChild(webview);
            webview = new MSWebView();
            Object.assign(this, {
                "webview": webview
            });
            webview.setAttribute("id", "WebView");

            // During startup our currentUrl field is blank. If the WebView has crashed 
            // and we were on a URI then we may obtain it from this property.
            if (browser.currentUrl && browser.currentUrl != "") {
                this.trigger("newWebview");
                this.navigateTo(browser.currentUrl);
            }
            webviewParent.appendChild(webview);

I run replaceWebView during startup to replace the in-process WebView created via HTML markup with an out-of-process WebView. I could be doing more to dynamically copy styles, attributes, etc but I know what I need to set on the WebView and just do that.

When a WebView process crashes the corresponding WebView object is no longer useful and a new WebView element must be created. In fact if the old WebView object is used it may throw and will no longer have valid state. Accordingly when the WebView crashes I run replaceWebView again. Additionally, I need to store the last URI we've navigated to (browser.currentUrl in the above) since the crashed WebView object won't know what URI it is on after it crashes.

            webview.addEventListener("MSWebViewProcessExited", () => { 
                if (browser.currentUrl === browser.lastCrashUrl) {                   ++browser.lastCrashUrlCrashCount; 
                } 
                else { 
                    browser.lastCrashUrl = browser.currentUrl; 
                    browser.lastCrashUrlCrashCount = 1; 
                } 
                // If we crash again and again on the same URI, maybe stop trying to load that URI. 
                if (browser.lastCrashUrlCrashCount >= 3) { 
                    browser.lastCrashUrl = ""; 
                    browser.lastCrashUrlCrashCount = 0; 
                    browser.currentUrl = browser.startPage; 
                } 
                this.replaceWebView(); 
            }); 

I also keep track of the last URI that we recovered and how many times we've recovered that same URI. If the same URI crashes more than 3 times in a row then I assume that it will keep happening and I navigate to the start URI instead.

The x-ms-webview HTML element has the void addWebAllowedObject(string name, any value) method and the webview XAML element has the void AddWebAllowedObject(String name, Object value) method. The object parameter is projected into the webview’s top-level HTML document’s script engine as a new property on the global object with property name set to the name parameter. It is not injected into the current document but rather it is projected during initialization of the next top-level HTML document to which the webview navigates.

Lifetime

If AddWebAllowedObject is called during a NavigationStarting event handler the object will be injected into the document resulting from the navigation corresponding to that event.

If AddWebAllowedObject is called outside of the NavigationStarting event handler it will apply to the navigation corresponding to the next explicit navigate method called on the webview or the navigation corresponding to the next NavigationStarting event handler that fires, whichever comes first.

To avoid this potential race, you should use AddWebAllowedObject in one of two ways: 1. During a NavigationStarting event handler, 2. Before calling a Navigate method and without returning to the main loop.

If called both before calling a navigate method and in the NavigationStarting event handler then the result is the aggregate of all those calls.

If called multiple times for the same document with the same name the last call wins and the previous are silently ignored.

If AddWebAllowedObject is called for a navigation and that navigation fails or redirects to a different URI, the AddWebAllowedObject call is silently ignored.

After successfully adding an object to a document, the object will no longer be projected once a navigation to a new document occurs.

WinRT access

If AddWebAllowedObject is called for a document with All WinRT access then projection will succeed and the object will be added.

If AddWebAllowedObject is called for a document which has a URI which has no declared WinRT access via ApplicationContentUriRules then Allow for web only WinRT access is given to that document.

If the document has Allow for web only WinRT access then projection will succeed only if the object’s runtimeclass has the Windows.Foundation.Metadata.AllowForWeb metadata attribute.

Object requirements

The object must implement the IAgileObject interface. Because the XAML and HTML webview elements run on ASTA view threads and the webview’s content’s JavaScript thread runs on another ASTA thread a developer should not create their non-agile runtimeclass on the view thread. To encourage end developers to do this correctly we require the object implements IAgileObject.

Property name

The name parameter must be a valid JavaScript property name, otherwise the call will fail silently. If the name is already a property name on the global object, that property is overwritten if the property is configurable. Non-configurable properties on the global object are not overwritten and the AddWebAllowedObject call fails silently. On success, the projected property is writable, configurable, and enumerable.

Errors

Some errors as described above fail silently. Other issues, such as lack of IAgileObject or lack of the AllowForWeb attribute result in an error in the JavaScript developer console.

You can call WinRT APIs from PowerShell. Here's a short example using the WinRT Launcher API:

[Windows.System.Launcher,Windows.System,ContentType=WindowsRuntime]
$uri = New-Object System.Uri "http://example.com/"
[Windows.System.Launcher]::LaunchUriAsync($uri)

Note that like using WinRT in .NET, you use the System.Uri .NET class instead of the Windows.Foundation.Uri WinRT class which is not projected and under the covers the system will convert the System.Uri to a Windows.Foundation.Uri.

You can use conditional breakpoints and debugging commands in windbg and cdb that together can amount to effectively patching a binary at runtime. This can be useful if you have symbols but you can't easily rebuild the binary. Or if the patch is small and the binary requires a great deal of time to rebuild.

Skipping code

If you want to skip a chunk of code you can set a breakpoint at the start address of the code to skip and set the breakpoint's command to change the instruction pointer register to point to the address at the end of the code to skip and go. Voila you're skipping over that code now. For example:

bp 0x6dd6879b "r @eip=0x6dd687c3 ; g"

Changing parameters

You may want to modify parameters or variables and this is simple of course. In the following example a conditional breakpoint ANDs out a bit from dwFlags. Now when we run its as if no one is passing in that flag.

bp wiwi!RelativeCrack "?? dwFlags &= 0xFDFFFFFF;g"

Slightly more difficult is to modify string values. If the new string length is the same size or smaller than the previous, you may be able to modify the string value in place. But if the string is longer or the string memory isn't writable, you'll need a new chunk of memory into which to write your new string. You can use .dvalloc to allocate some memory and ezu to write a string into the newly allocated memory. In the following example I then overwrite the register containing the parameter I want to modify:

.dvalloc 100
ezu 000002a9`d4eb0000 "mfcore.dll"
r rcx = 000002a9`d4eb0000

Calling functions

You can also use .call to actually make new calls to methods or functions. Read more about that on the Old New Thing: Stupid debugger tricks: Calling functions and methods. Again, all of this can be used in a breakpoint command to effectively patch a binary.

2016-Nov-5: Updated post on using Let's Encrypt with NearlyFreeSpeech.net

I use NearlyFreeSpeech.net for my webhosting for my personal website and I've just finished setting up TLS via Let's Encrypt. The process was slightly more complicated than what you'd like from Let's Encrypt. So for those interested in doing the same on NearlyFreeSpeech.net, I've taken the following notes.

The standard Let's Encrypt client requires su/sudo access which is not available on NearlyFreeSpeech.net's servers. Additionally NFSN's webserver doesn't have any Let's Encrypt plugins installed. So I used the Let's Encrypt Without Sudo client. I followed the instructions listed on the tool's page with the addition of providing the "--file-based" parameter to sign_csr.py.

One thing the script doesn't produce is the chain file. But this topic "Let's Encrypt - Quick HOWTO for NSFN" covers how to obtain that:

curl -o domain.chn https://letsencrypt.org/certs/lets-encrypt-x1-cross-signed.pem

Now that you have all the required files, on your NFSN server make the directory /home/protected/ssl and copy your files into it. This is described in the NFSN topic provide certificates to NFSN. After copying the files and setting their permissions as described in the previous link you submit an assistance request. For me it was only 15 minutes later that everything was setup.

After enabling HTTPS I wanted to have all HTTP requests redirect to HTTPS. The normal Apache documentation on how to do this doesn't work on NFSN servers. Instead the NFSN FAQ describes it in "redirect http to https and HSTS". You use the X-Forwarded-Proto instead of the HTTPS variable because of how NFSN's virtual hosting is setup.

RewriteEngine on
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [L,R=301]

Turning on HSTS is as simple as adding the HSTS HTTP header. However, the description in the above link didn't work because my site's NFSN realm isn't on the latest Apache yet. Instead I added the following to my .htaccess. After I'm comfortable with everything working well for a few days I'll start turning up the max-age to the recommended minimum value of 180 days.

Header set Strict-Transport-Security "max-age=3600;" 

Finally, to turn on CSP I started up Fiddler with my CSP Fiddler extension. It allows me to determine the most restrictive CSP rules I could apply and still have all resources on my page load. From there I found and removed inline script and some content loaded via http and otherwise continued tweaking my site and CSP rules.

After I was done I checked out my site on SSL Lab's SSL Test to see what I might have done wrong or needed improving. The first time I went through these steps I hadn't included the chain file which the SSL Test told me about. I was able to add that file to the same files I had already previously generated from the Let's Encrypt client and do another NFSN assistance request and 15 minutes later the SSL Test had upgraded me from 'B' to 'A'.