Test Driven Game Development (TDGD)

[gin5eng]

Hey everyone,

I've been getting into TDD (Test Driven Development) at work and I've seen how it can be very valuable to a project during development and later on during maintenance.

 

So I was curious, has anyone applied TDD to game development? If you you have any examples or tidbits that you can share that would be awesome!

[rich]

For apps (and web sites) it's extremely useful, but for games it's next to impossible to implement well - they're so dynamic, rely so much on user input and specific event timing that unless your game is dead simple or very fixed, like a card game for example with predetermined AI, it's really hard to fit TDD into that.

[benny!]

Personally, I would say that it depends on the size of the project. 

 

If you are working on a bigger project with multiple programmers, I highly recommend test driven development.

 

For apps (and web sites) it's extremely useful, but for games it's next to impossible to implement well - they're so dynamic, rely so much on user input and specific event timing that unless your game is dead simple or very fixed, like a card game for example with predetermined AI, it's really hard to fit TDD into that.

 

Well, concerning user input/interaction you may be right. Even though various systems exist simulating user input - I agree that it might be hard and in some cases not worth the effort. However, each game has some game logic which can be tested (automatically). This logic is the same as business logic in "normal" apps - so I do not see a difference between a game and a "normal" app here.

[rich]

It's the potential number of permutations that make the difference - a business app nearly always has a fixed route, a required set of data expected in a transaction - it's painfully easy to write cases that test for the validity (and existence) of said data, as well as what happens to it when it is processed. Only occasionally is the outcome based on constantly changing data from another source.

 

In most (but not all) games it's the opposite. Writing test cases to check game engine logic is easy and valuable. Writing them to test if the game is playing correctly is, imho, a waste of time given the sheer number of possible outcomes. But as I said this is game specific.

[Quetzacotl]

I'm  now writing game using TDD (as I do in all my projects) and I find it extremely useful. But to do it right we don't test gameplay mechanics, because there is too many routes, as you said earlier. We test every piece of game separately. Tests should run and pass in encapsulated enviroment, if they depends on ten other pieces of game engine, then your test or your engine is done wrong. For example test

check that if player A shoots bullet and this bullet hits enemy B, then enemy B lose HP

is really testing of few pieces separately: class Player, class Bullet, collision checks, class Enemy and most probably few more.

If these tests are done right  then we don't have to make tests of gameplay with user inputs, because it's redundant.

 

Another thing about TDD and actually the most important which you didn't mention yet, is purpose of TDD, not tests alone but Test Driven Development.

When you write tests first you are in fact designing your game engine and that way you can avoid many problems which you would encounter by implementing code first.

Code wrote with TDD has much, much higher quality then code without it and solely for this reason you should use TDD everywhere you can.

[gin5eng]

Awesome, so far I think everyone has posted very valid points.

 

@Quetzacotl

It sounds like your also talking about integration testing. (check that if player A shoots bullet and this bullet hits enemy B, then enemy B lose HP) You could also have unit tests that would test to see if a player CAN shoot and CAN loose HP if damaged. I especially agree with your last paragraph too.

 

I would also like to add that the tests from TDD can act as a test harness for large projects. So that changes to core elements can be made while making absolutely sure that the game still works as it did by looking at the tests after the changes. I guess this is also a form of regression testing.

[hima]

I'd love to apply TDD to game as well, but I feel the same as Rich; game is just too dynamic. 

 

Also, I feel like a small HTML5 game doesn't really required writing tests since it might take more time than you really need to. For a medium to large size project, maybe.

[Quetzacotl]

Not using tests is a technical debt.  http://en.wikipedia.org/wiki/Technical_debt  You can make this debt if you are not sure that your game is playable and fun, and you want to test it fast (prototyping). But if game is OK then you have to pay this debt sooner or later or you will get into many troubles.

[rich]

I'd wager for the vast majority of web games there is minimal technical debt because they never grow large enough, or need to be maintained beyond release, to ever come close to needing to pay it off.

 

For frameworks? Yes. For multiplayer or large and complex games? Yes again. For the remaining 99%, it's time wasted that would be better spent fine tuning gameplay.

[Quetzacotl]

I'd wager for the vast majority of web games there is minimal technical debt because they never grow large enough, or need to be maintained beyond release, to ever come close to needing to pay it off.

 

For frameworks? Yes. For multiplayer or large and complex games? Yes again. For the remaining 99%, it's time wasted that would be better spent fine tuning gameplay.

this is valid if you think "ok, I'll make a game, but I don't want to earn money or make this game famous, I just to want to fail and get some experience". Be honest, most of the time if you are creating a game, no matter what, you want it to success. But you can't success if your game is full of annoying bugs.

 

Another thing is that right now it's hard to get a profit from web games because there aren't many (if any) platforms to sell it.

 

One more thing. When you create game and you care about encapsulation and TDD. Then you can  easly reuse your code in next game. If you don't, then most probably you will need to tweak your code, and tweaking means bring to mind "how was it working again?". So you are reimplementing it again and wasting time.

 

I know using TDD is hard. You won't get it the first time you use it, probably you won't get it even second and third time. But with next project you will finally understand why tests should be unavoidable in any project. TBH for me people that are not using tests are just lazy.

[rich]

TDD does not mean your game is bug free, scalable, easily portable or profitable at all. Equally not using a TDD approach absolutely in no way means your game will contain bugs either.

 

Writing  tests involves human input into which human error creeps. Games are constantly refactored during development, often with whole sections added or removed or  whole mechanics tweaked to get the feeling right. Add to this the visual nature of games, something TDD cannot test easily, and the time spent constantly updating the tests is a technical debt of another kind. Factor in that most bugs in the wild are hardware/platform/browser manifestations rather than code logic and the workload multiplies.

 

TDD has it's place - mostly in the heart/core of the game or libraries used by it - for example a physics or collision system without tests would be insane, as you'd never know if it was the math screwing up or something else entirely. But it's not a holy grail and I've witnessed it delay projects as much as it benefited them, because they tried applying it globally to the entire game rather than only in the areas in which it made sense. Use it, yes, but use it intelligently.

[benny!]

... Use it, yes, but use it intelligently.

 

Trve. "Unit Tests" are just one of many tools out there that help you in your project. According to my experience it is important to create a custom  toolset/toolchain for each project, since each project has its own requirements.

 

Speaking of Unit Tests, I recommend to setup project specific rules for each project defining what to test and how to test. You need to find a good economic balance there (time/costs/effectiveness etc).

[Quetzacotl]

TDD does not mean your game is bug free, scalable, easily portable or profitable at all. Equally not using a TDD approach absolutely in no way means your game will contain bugs either.

 

Writing  tests involves human input into which human error creeps. Games are constantly refactored during development, often with whole sections added or removed or  whole mechanics tweaked to get the feeling right. Add to this the visual nature of games, something TDD cannot test easily, and the time spent constantly updating the tests is a technical debt of another kind. Factor in that most bugs in the wild are hardware/platform/browser manifestations rather than code logic and the workload multiplies.

 

TDD has it's place - mostly in the heart/core of the game or libraries used by it - for example a physics or collision system without tests would be insane, as you'd never know if it was the math screwing up or something else entirely. But it's not a holy grail and I've witnessed it delay projects as much as it benefited them, because they tried applying it globally to the entire game rather than only in the areas in which it made sense. Use it, yes, but use it intelligently.

I agree with you. I didn't say test everything. Tests of core engine is enough and should cover most of issues. If we talk in design patterns, all I test is model, I don't try to test controller or view because there is too much overhead to it. 

 

But core(models) should be testes always.

[rich]

Now would be a good point for everyone to list their favourite unit testing libraries

[Quetzacotl]

I'm using QUnit, it's fine, but didn't try anything else yet.

[benny!]

Javascript wise I also used QUint for two projects so far. Also had a quick test drive with JS-Test-Driver some years ago.

 

If it comes to PHP I used SimpleTest but switched to PHPUnit.

[Quetzacotl]

http://devopsreactions.tumblr.com/post/44444575875/when-we-dont-write-a-test-for-that-special-corner-case