Freitag, 11. Dezember 2015

Advanced NCrunch: Isolating tests

I am constantly amazed on how far you can push the frontiers of automated testing in order to reduce blind spots and mitigate software risks.

For instance, my friend Thomas (@ThomasMentzel) currently works on automating screenshots for continuous documentation in the context of a standard test runner like NUnit. I am sure he will post about this soon.

This is just one example on how to easily extend automated testing on aspects most people would not consider for unit testing. And yeah, right - testing behaviour on external resources like the file system (tathamoddie/System.IO.Abstractions), the registry (jozefizso/SystemWrapper), etc. belongs to integration testing and is outside the scope of unit testing, strictly spoken.

However, test runner frameworks like NUnit, xUnit, etc. combined with continuous testing tools like NCrunch are so powerful & easy to use that i don't see any reason why not to formulate these kind of tests as simple unit tests, e.g. using the Explicit or Category attribute.

Today, i want to share a use case for some lesser known features of NCrunch that allow you to tweak "how isolated" your tests run.

A while ago, i posted about my experiences on handling native dependencies re-introducing an ancient trick, i.e. modifying your process environment by injecting a customized PATH environment variable in the application bootstrapping phase.

Until recently, i neither needed to modify nor test the bootstrapper code. But then a change request came in, that forced me to touch it. In this particular case, we wanted to simplify Oracle connection strings by using tnsnames.ora which required to optionally override the TNS_ADMIN environment variable as well.

Following TDD's red-green-refactor cycle i wanted to start a "red" - i.e. failing - test. But how can you isolate System.Environment without affecting all other test cases?

This is where the NCrunch IsolatedAttribute comes in. The cool thing about it is that you don't even need to include a reference to the NCrunch framework, because Remco Mulder (@remcomulder), NCrunch's main developer, made it super easy to use it.

Here is what the - now "green" - test looks like. It includes testing that the bootstrapper logs a bit about what he does.

Needless to say, that the implementation did not require any more manual testing.

[Test(Description = "#: simple oracle db connection (using TNS_ADMIN)")]
// Since we check that environment variable will be set correctly, 
// we need to isolate this test
// cf.:
public void Bootstrap_oracle_and_tns_admin()
    var config = new Configuration();
        .Should().Be("%ORACLE_HOME%", "should be default");
    config.TnsAdmin.Should().Be("%ORACLE_HOME%", "should be default");

    var expandedOracleClientPath = Environment

    var expandedTnsAdmin = Environment


    // cf. github NEdifis
    using (var ttl = new TestTraceListener())
      var sut = new OracleModule(config);
      // // extension method, cf. github AutoFac.TestingHelpers

      var infos = ttl.MessagesFor(TraceLevel.Info).ToList();
        $"Using Oracle connection string: '{sut.OracleConnectionString}'.");
        $"Using '{expandedOracleClientPath}' to lookup Oracle native libraries.");
        $"Using TNS_ADMIN='{expandedTnsAdmin}' to lookup 'tnsnames.ora'.");