Many people don’t know about how awesome the windows debuggers are. I recently got a question from a volunteer mentee: he was experiencing a startup crash in Firefox and he wanted to know how to get the debugger attached to Firefox before the crash.

On other systems, I’d say to use mach debug, but that currently doesn’t do useful things on Windows. But it’s still pretty simple. You have two options:

Debug Using Your IDE

Both Visual Studio and Visual C++ Express have a command-line option for launching the IDE ready for debugging.

devenv.exe -debugexe obj-ff-debug/dist/bin/firefox.exe -profile /c/builds/test-profile -no-remote

The -debugexe flag informs the IDE to load your Firefox build with the command lines you specify. Firefox will launch with the “Go” command (F5).

For Visual C++ express edition, run WDExpress.exe instead of devenv.exe.

Debug Using Windbg

windbg is a the Windows command-line debugger. As with any command-line debugger it has an arcane debugging syntax, but it is very powerful.

Launching Firefox with windbg doesn’t require any flags at all:

windbg.exe obj-ff-debug/dist/bin/firefox.exe -profile /c/builds/test-profile -no-remote

Debugging Firefox Release Builds

You can also debug Firefox release builds on Windows! Mozilla runs a symbol server that allows you to automatically download the debugging symbols for recent prerelease builds (I think we keep 30 days of nightly/aurora symbols) and all release builds. See the Mozilla Developer Network article for detailed instructions.

Debugging official builds can be a bit confusing due to inlining, reordering, and other compiler optimizations. I often find myself looking at the disassembly view of a function rather than the source view in order to understand what exactly is going on. Also note that if you are planning on debugging a release build, you probably want to disable automatic crash reporting by setting MOZ_CRASHREPORTER_DISABLE=1 in your environment.

Stuart has been working on using an allocator for Mozilla which has much better performance characteristics, especially with memory fragmentation and heap growth over time. The allocator he chose is jemalloc, the default allocator for the FreeBSD libc. On Linux, intercepting and replacing malloc is fairly easy, because of the way dynamic symbol loading works. On Windows, however, it is difficult or impossible to intercept and redirect calls to malloc to a custom allocator. So instead of trying to hook to a prebuilt CRT, I spent most of today hacking the Windows C runtime (CRT), replacing the default allocator with jemalloc.

The Windows CRT sources come with VC8 professional edition so any licensed user may hack on them and redistribute the result as part of a larger program. It comes with a mostly-working nmakefile¹: I had to disable the code which builds the managed-code CRT because apparently I don’t have the right .NET headers installed. Getting the new jemalloc.c file to build wasn’t that hard, either: it required a few #defines, typedefs, and disabled warnings, but nothing serious. The hardest part about replacing It was figuring out what parts of the original CRT were heap-related and removing them correctly. It was a wild ride, but I think that I have a build of the Windows CRT that works… at least small programs like xpidl and shlibsign work.

Unfortunately, according to the EULA² I am not allowed to redistribute this modified CRT by itself. So the only way you can get it is by distributing it with a Firefox build. Also, I’m not allowed to post the patch queue which I used to develop the custom CRT, because those patches may contain copyrighted code in context. Do any of my readers know of a format that will alter a set of files according to a set of instructions without the instructions revealing the contents of the original files? I would really love it if Microsoft would release their C runtime code under a liberal open-source license… can someone suggest a good person to contact at Microsoft?

Stuart will have some builds posted soon, once a few kinks get ironed out.

For Mozilla2 we’re probably going to push the solution to an intermediary library: we will have a single allocator library which is used for both garbage-collected (managed) and explicitly allocated/freed (unmanaged) memory. We will switch back to the standard CRT, but we will try to avoid using the standard CRT allocator at all. See the “space management” thread on the tamarin-devel mailing list (December and January) for some background discussion.

1. nmake is one of the suckiest build systems on the planet. I could get better results with a .vbs.
2. From the EULA, section 3.1: …you agree: (i) except as otherwise noted in Section 2.1 (Sample Code), to distribute the Redistributables only in object code form and in conjunction with and as a part of a software application product developed by you that adds significant and primary functionality to the Redistributables…