We came across an interesting problem in SymbolSource recently. In the current architecture, the support for NuGet and OpenWrap is provided though so called gateways, that act as protocol adapters connecting to our SOAP API. In theory this provides a great testing opportunity, because it is easy to mock a set of relatively simple, procedural APIs.

A lot has been written about testing MVC applications: that this paradigm of programming is unit test friendly, because you can test the controller in isolation, or you could even try testing views. But what about configuration? Routing? Handlers? That can only be tested in an environment with a fully running web application. Because of this, I should probably have put integration testing in the post’s title, but since we wanted to focus on fully automating our gateway tests and running them with xUnit in CI, let’s keep the original term.

Step 1: running a web application in a unit test

Fortunately, still is quite easy thanks to the hosting APIs exposed in System.Web.Hosting, and even easier with the CassiniDev project (also available on NuGet). It is a fork of the original Visual Studio Development Web Server codenamed Cassini, with a new UI and a lot of other improvements. Of course you can skip the UI altogether and integrate with its APIs. Yes, Cassini is far from truly emulating IIS, or even IIS Express, but it’s still much better than having to test manually, or create automation that does a full deployment on a server.

This is what we came up with at first as a test base class:

public class GatewayTestsBase : IDisposable
{
    private readonly CassiniDevServer server;

    public GatewayTestsBase()
    {
        server = new CassiniDevServer();
        server.StartServer("C:\SymbolSource\SymbolSource.Gateway.NuGet");
    }

    public void Dispose()
    {
        server.StopServer();
        server.Dispose();
    }

    protected string SmokeTest(string path)
    {
        var url = server.NormalizeUrl(path);
        Debug.WriteLine("Testing: " + url);

        try
        {
            using (var client = new WebClient())
                Debug.WriteLine(client.DownloadString(url).Trim());
        }
        catch (WebException exception)
        {
            using (var stream = exception.Response.GetResponseStream())
            using (var reader = new StreamReader(stream))
                Debug.WriteLine(reader.ReadToEnd().Trim());

            throw;
        }

        return url;
    }
}

SmokeTest?

Yes, we thought it would be a good idea to issue a blank request (if applicable) to any URL we’re about to test, to see if the application starts at all (configuration errors, connectivity issues, etc). So most of our test classes start with something like this:

[Fact]
public void SmokeTest()
{
    SmokeTest("/");
}

What about self-containment?

Ah, yes, an excellent question. You must have noticed the hardcoded path to the web root (“C:\SymbolSource\SymbolSource.Gateway.NuGet”). For now we solved it by disabling xUnit’s shadow copying of test assemblies, which causes it to run tests from %LocalAppData%\Temp instead of the build location, and a change to test setup like this:

public class GatewayTestsBase
{
    public GatewayTestsBase(Assembly appAssembly)
    {
        var testAssembly = GetType().Assembly;

        var appRoot = testAssembly.Location
            .Split(Path.DirectorySeparatorChar)
            .TakeWhile(name => name != testAssembly.GetName().Name)
            .Concat(new[] { appAssembly.GetName().Name })
            .Join(Path.DirectorySeparatorChar.ToString());

        server = new CassiniDevServer();
        server.StartServer(appRoot);
    }
}

public static class StringExtensions
{
    public static string Join(this IEnumerable values, string separator)
    {
        return string.Join(separator, values);
    }
}

What’s appAssembly?

GatewayTestsBase is shared between test projects for NuGet and OpenWrap (which of course are separate web applications), so in each test class we pass a reference to the assembly (web application) under test. Having a project reference also forces it to build as dependency of the test project. The idea behind the LINQ that build the appRoot path is to get from:

C:\SymbolSource\SymbolSource.Gateway.NuGet.Tests\bin\Release\SymbolSource.Gateway.NuGet.Tests.dll

to:

C:\SymbolSource\SymbolSource.Gateway.NuGet

A good improvement (and good test practice) would be to copy the web application to a temporary folder and apply any needed web.config changes before actually starting it. That eliminates any possibility of tests affecting each other.

Step 2: mocking an object and redirecting it back to the test assembly

As mentioned before, the web application under test is only an adapter that calls back into our server API. For each test we wanted to setup a different mock object for the API class and have the application call into it. What makes this hard is that the only connection with the application, that we have in our test, is through the public HTTP interface and through the file system. And it runs in a different app domain. The solution that we came up with is to expose the mock with .NET Remoting and add a bit of support to the application itself, so that redirection can be configured in web.config. This will be described in part 2. Part 3 will be about exercising the HTTP interface, which we are doing by automating nuget.exe and o.exe.

Stay tuned for more!