As promised, this post will present the results of my investigation into the performance impact of running under the simplest possible debugger written using MDbgEngine.

Timing method

For timing of all the samples, I used this method inside the debugee:

private static void Time(Action action, int count)
{
    var sum = 0L;

    for (var i = 0; i < count; i++)
    {
        var s = new Stopwatch();
        s.Start();
        action();
        s.Stop();
        sum += s.ElapsedMilliseconds;
        Thread.Sleep(1000);
    }
    Console.WriteLine(sum/count);
}

To minimize the impact of random system events, I run all of the samples 10 times, with a 1 second pause between each invocation, and then took an arithmetic average of the execution times.

CPU-bound code

private static void CPU(int count)
{
    Array.Sort(Enumerable.Range(1, count).Reverse().ToArray());
}

Conclusion: Running under a debugger has no impact on CPU-bound code. All measurements are within the margin of error.

IO-bound code

Base on the above findings, and a personal hunch, I decided to skip this one.

Exception-bound code

This test should prove most interesting, as it’s the only that has code really interfacing with the debugger.

The sample code throws exceptions in a loop:

private static void Exception(int count)
{
    for (int i = 0; i < count; i++)
    {
        try
        {
            throw new Exception();
        }
        catch
        {
        }
    }
}

And the debugger inspects them and does nothing:

process.PostDebugEvent +=
    (sender, e) =>
        {
            if (e.CallbackType == ManagedCallbackType.OnBreakpoint)
                process.Go();

            if (e.CallbackType == ManagedCallbackType.OnException2)
            {
                var e2 = (CorException2EventArgs)e.CallbackArgs;
            }

            if (e.CallbackType == ManagedCallbackType.OnProcessExit)
                stop.Set();
        };

Conclusion: Going back and forth between the debugger and its debuggee, causes a performance drop of 2 orders of magnitude. I guess Visual Studio is more because of the Output window, which logs all exceptions.

The question is: how many exceptions does code generate on average? Instead of guesstimating, I decided to guessmeasure. All of the code indexed by SymbolSource, including only the most recent version of each project, consists of more than 1.5 million lines of code (that’s where I stopped counting). Of the lines used for this measurement, there were 12 thousand lines that contained the word “throw”. Therefore, I have guessmeasured, that all code is 0.8% exception on average.

Final conclusion: running under MDbgEngine causes a 24.8% drop in performance.

Would you be willing to take that, for the promise of gaining access to detailed call stacks and being able to generate minidumps for really hard to catch bugs?

Personally, I’ll shout a loud yes for all development and integration, and for occasional production too.