Unit Testing

Every now and then there comes a time in a project when it would be nice to be able to write a unit test for a new or already existing feature. The reason could be that the feature is quite complex and we like to make sure to get the basics bug-free before going on creating all the stuff around it. Or you have an existing feature that doesn’t really work the way it should work and it is pretty tedious to find the bug in the running game.

Two extremes

Although unit testing already exists for quite a long time, there is still no standard way to use them in game projects. There are two extremes happening in projects: Using no unit tests at all and testing every single bit (test-driven development).

As it is with extremes the best way for a specific project most of the time lies between them. A really simple project might get along with no tests at all while test-driven development could make sense for a framework or a project with a real big team.

For me it is great to have the possibility to set up a unit test quickly but not being forced to add one for every single bit of code. This means the infrastructure has to be there and everything should run as automated as possible.

Unit testing for Unity

It seems like the devs at Unity Technologies  really got into unit testing last year. There were several nice blog posts published as well as the official Unity Test Tools asset. The blog posts are about unit testing in general, about NSubstitute and a special one about testing MonoBehaviours.

The conclusion one has to make after reading the blog posts though is that you should try to get your game logic out of Unity to test it properly. If your tests don’t rely on Unity they will be much faster to execute and more stable. More often though separating the code to test from Unity is the only way to create a unit test for it anyway.

If you really like to test specific game objects inside Unity, the Unity Test Tools can help you. Personally I have never really used this as it tests more of the engine than my own code. I really prefer to isolate the code to test in a nice way, so it doesn’t depend on Unity anymore.

Setting up your tests

When you start to write unit tests, the first thing to do is creating an own test project for each project you want to test. In Visual Studio you will have one test project for every game project file. These test projects will reference the game project of course as well as your unit testing framework (probably NUnit).

For the folder structure in the test project you can just copy the one from the project you test. The class names could be “ClassNameTest”. This makes it easy to find the tests for a specific class.

Automated testing on Jenkins

Once you wrote unit tests you want to make sure they stay valid. Even more you want to be informed immediately when one fails, so you know which code change broke it.

If you set up a build server for your game you should use it to run the unit tests for the projects it creates right after they were built. Most useful is the unit testing after the continuous job which is built every time something in the game code changed. This way you will get informed directly after a dev added/changed code that breaks a test and it’s quite easy to fix it then (either the test or the new code).

Setting up Jenkins is easy as there are several good and active plugins. The results of the unit tests are visualized in a quite nice way, also adding some stats to check how the number of tests changed and how often tests failed. You’ll also need to install NUnit on your build server, of course. Here are some step-by-step instructions.

Deciding when to write a unit test

This questions bothered me quite some times. I haven’t got a definitive answer for it though. This is related to the pros and cons of unit tests:

Pros:

  • Makes sure your tested code delivers the results you expect it to do
  • Makes sure your tested code stays valid, even after other people changed it
  • You can refactor your code without worrying too much to change its functionality
  • You can test special cases that are not easily to reproduce in a running game
  • Writing a unit test forces you to isolate the part of code you want to test which (probably) improves the structure of your project

Cons:

  • Writing unit tests is some work, especially setting up the code in a way that it can be easily tested isolated by the rest of your game
  • Maintain unit tests for code which functionality changes is much more work

Especially game projects change quite a lot over time. So having a lot of unit tests means that you have to maintain the unit tests as well. Plus there will be unit tests that get obsolete when some functionality is removed from the game.

That’s why there are only two cases when I personally write unit tests for a feature:

  • The feature is quite complex. This means writing some tests for it upfront (as in test driven development) will make sure it behaves the way it should even before integrating it into the game itself.
  • There is a (hard to reproduce) bug. Writing a unit test for it probably will save some time fixing the bug as you can easily trigger it with the test. After you found the reason for the bug and fixed it, you just run the test once more and it should succeed. A further advantage is that this exact bug won’t never occur again (or rather be detected immediately) as there is a unit test for it now.

Bonus: Check out NSubstitute

If you are already into unit testing you should definitively try out NSubstitute. It allows to mock up interfaces in your unit tests to let them return the values with which you like to test.

This means you can easily isolate the code you want to test and don’t have to setup your whole project. E.g. you might have a class which depends on an external service. If you pass in this service via an interface you can use a dummy service in a unit test instead. This allows also testing situations that are hard or impossible to test otherwise, like the service returning a specific error response.

Conclusion

Unit testing is a nice tool to have in software development and the frameworks like NUnit are pretty solid. The extra time to set it up in your project is pretty small and definitively worth it.

When and for what to write unit tests is up to you and your team to decide and depends on the project. For my projects there were never any rules when to write a unit tests, but it was up to each developer to decide. In the end you have to realize when it is worth to write some tests and when it isn’t. This requires some experience as does writing the tests themselves.

One last warning: Starting with unit testing as an inexperienced developer might be a hard lesson depending on your current code structure. It requires a quite modular code architecture to be able to test parts of your project separately. So before you gain all the other benefits of unit testing, the first one is probably a forced improved coding style. Which isn’t that bad either, is it? 🙂

  • Just wanted to leave a small hint on TeamCity. I’ve learned to love it. I found it to be way easier to set up and to maintain projects, plugins and build agents.

    Just as a small side note: it’s free until you really use it to an extreme level – https://www.jetbrains.com/teamcity/