ORIGINAL: Shannon V. OKeets
ORIGINAL: Dabrion
ORIGINAL: paulderynck
The most frustrating thing is that many bug fixes seem to generate other bugs - often in game situations (FREX Multiple States of War ) or option sets that no one has time to try. Add to that if you are doing an update a week, the beta testers (all of whom actually have real lives to lead) have about 48 hours to test things.
Trying to code a game with so many complicated rules, exceptions and options - and stay as true as possible to the rules - makes MWiF "A Subway Named Mobius".
That's what makes it so different from strategic games designed for a computer where the designer controls everything he wants to simulate and can simply toss out some especially complex-to-code feature or option he'd thought of putting in originally - because nobody knows it would violate rule XYZ.
Complexity and coupling of code passages is not intrinsic to wargames. There are means to address this. I find the "it's so incredibly hard, it can never be done" tune to be a sad excuse for not using modern software engineering and agile development techniques.
The issue here is not with the problem domain or the solution itself. If you had unit tests, you would immediately know if that new code change impacts code written earlier. You also could automate 80% of what is called "beta testing" in MWiF terms. I can see how this all is a very frustrating experience, monkey patching code all the time.. doesn't have to be that way..
"Unit Test", what a great phrase! Could you give one example of it for MWIF? That shouldn't be difficult since you feel sure that 80% of the code could be automated for testing using "unit tests".
By the way, I have no interest in a theoretical definition - I live and work in a world of practical applications.
As a professional expert Software Architect and programmer, my opinion is that unit test are only good in a handful of cases. Theoretical books about it are usually writen by people who don't do practical, industrial programming.
Unit testing logically requires testing each and every function against all possible input. That's just not possible considering that there are far more different input combinations than their are stars in the sky... So the next step is to reduce the number of inputs so that at least each branch in the tested code is run. That's what is considered the minimal unit test and it is usually already difficult enough to implement correctly. For exemple, you are supposed to test code that catches errors that are not even supposed even to happen... So you have to bring up some ways to generate these errors.
All this to say that writing a correct set of unit tests takes far more than twice the time it took to write the original code.
Now, in addition, the methods you test don't live in limbo : they are in the program environment, which itself may change the code behaviour. The most obvious case for MWIF are optional rules. A good unit test should at least minimally check each method against all possible combinations of optional rules. I did not count, but I believe that's more than 60. Even if you can reduce the number of relevant options that could possibly affect the function,you'll likely get more than ten in most cases. That just about a thousand cases...
Last, but not least, unit testing has its own limits: every time you change a method, you have to update the tests. Every time you split a method, you have to discard the old tests and write new tests for two methods...
The net result of relying solely on extensive unit testing is that very soon, everytime you code, you feel like youare glued: for even a simple change in a method, which you logically know is correct, you may have to adapt hundreds of lines of test results,if not the tests themselves.
Well well...
All this to say that unit testing is far from being the alpha and omega of writing correct and resilient code. Relying on a huge batch of unit test is a poor way for discarding one's own errors to the next programmer. It doesn't warrant solid software architecture and actually hinders any late change to the architecture, considering that heavy change means dropping thousand of lines of test code.
Another drawback to entranched unit testing is that, for easily understood reasons, programmers then tend to write huge methods, which goes against simplicity and reuse.
On my part, my methods usually rarely exceed five line of code. Often,they are just one line or two. Unit testing for that ? Come on!
A last word: what I say must not even be read as a is. Suppose I am writing sofware for running a plane or a nuclear plant. In that case I would insist on extensive unit tests, well chosen (not all methods, but critical methods such as that one that recovers the input from sensor X or Y, which may in turn use utility methods) ; it would carry a hefty cost of course ; but that cost would be tied with the security side of the project.
Just my two cents...
Yves
